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Abstract 

Linear codes with good joint spectra are good candidates for lossless joint source-channel coding (JSCC). However, in spite 
of their existence, it is still unknown how to construct them in practice. This paper is devoted to the construction of such codes. 
In particular, two families of linear codes are presented and proved to have good joint spectra. The first family is derived from 
Gabidulin codes, a class of maximum-rank-distance (MRD) codes. The second is constructed by a serial concatenation of an 
inner low-density generator matrix (LDGM) code and an outer low-density parity-check (LDPC) code. The general criteria of 
good linear codes are also reviewed and discussed for three cases: lossless source coding, channel coding, and lossless JSCC, 
respectively. In the framework of code-spectrum approach, it is shown that all these issues just correspond to the problems of 
constructing linear codes with good kernel spectra, good image spectra, and good joint spectra, respectively. Furthermore, it is 
shown that good joint spectra imply good kernel spectra and good image spectra. This fact thus establishes the fundamental role 
of linear codes with good joint spectra. Conversely, it is proved that, when the output (resp., input) alphabet is an elementary 
abelian group, there exist linear codes with not only good joint spectra but also the same kernel (resp., image) of any given linear 
code that has a good kernel (resp., image) spectrum. 

Index Terms 

Code spectra, Gabidulin codes, linear codes, low-density generator matrix (LDGM) codes, low-density parity-check (LDPC) 
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Constructing Linear Codes with Good Spectra 



I. Introduction 

Linear codes, owing to their good structure, are widely 
applied in the areas of channel coding, source coding, and 
joint source-channel coding (JSCC). Undoubtedly, construct- 
ing good linear codes is always one of the most important 
and everlasting topics in these areas. As we know, a variety 
of good linear codes such as Turbo codes [4] and low-density 
parity-check (LDPC) codes [12|, |22| have been constructed 
for channel coding. And in the past decade, the parity-check 
matrices of good linear codes for channel coding have been 
employed as encoders for distributed source coding. They 
proved good in both theory EH, (39l and practice |5l. ll29l, 
|34|. However, in the general cases of lossless JSCC (based on 
linear codes), it is still unknown what kinds of linear codes are 
good and how to construct them. We do not even know how 
to design an implementable optimal JSCC scheme based on 
linear codes for arbitrary sources and channels. For instance, 
much work on practical designs of lossless JSCC based on 
linear codes has been done for transmission of specific corre- 
lated sources over specific multiple-access channels (MACs), 
e.g., correlated sources over separated noisy channels fl4l . 
1 40 1, 1 42], correlated sources over additive white Gaussian 
noise (AWGN) MACs 021, [25 1, and correlated sources over 
Rayleigh fading MACs 1411 . but it is still not clear how to 
construct an implementable optimal lossless JSCC scheme for 
a general case. The same problem also exists in the case of 
point-to-point transmission, but since traditional (nonlinear) 
source coding techniques combined with joint source-channel 
decoding work well in this case, linear-code-based schemes 
are not so important as in distributed JSCC. One exception is 
when we need a simple universal encoder, that is, the encoder 
does not require any knowledge of the source statistics. For 
background information of lossless JSCC in the point-to-point 
case, we refer to 11201 . ||431 and the references therein. 

Recently, for lossless transmission of correlated sources 



over MACs, we proposed a general scheme based on linear 
codes (371, which was proved to be optimal if good linear 
codes and good conditional probabilities are chosen. Fig. [T] 
illustrates the mechanism of the encoding scheme (see l37l 
Sec. III-C]), which can also be formulated as 

*n(v) = g n (v,S i (F„(E„(v))) + U l ) Vv e V". 

Roughly speaking, the scheme consists of two steps. First, the 
source sequence v is processed by a random affine mapping, 
i.e, (Ejo.F n oE n )(v) + L/ ! , where F n is a random linear code, 
E/ and E n are independent uniform random interleavers, and 
U l is an independent uniform random vector. Secondly, the 
output of the first step, together with the source sequence, is 
fed into the quantization mapping q n to yield the final output. 
The first step is to generate uniformly distributed output 
with the so-called pairwise-independence property, while the 
second step is to shape the output so that it is suitable for a 
given channel. Accordingly, two main issues arise: how do we 
design a good linear code F n and a good quantization mapping 
q n which fulfill the above two goals, respectively? About the 
former, we proved in ll37l that linear codes with good joint 
spectra (a generalization of input-output weight distributions) 
are good candidates. Hence, to design a lossless JSCC scheme 
in practice, a big problem is how to construct linear codes with 
good joint spectra. To our knowledge, however, this problem 
has never been studied before. 

In this paper, we shall give a thorough investigation of this 
problem. Our main tool is the code-spectrum approach |37|. 
As we shall see, the spectra of a linear code, including kernel 
spectrum, image spectrum, and especially joint spectrum, 
provide important characterization on its performance for most 
applications. The rest of this paper is organized as follows. 

In Section [Til we introduce the basic notations and con- 
ventions used throughout the paper. In Section Hill we briefly 
review the code-spectrum approach. Then in Section [IV] we 
develop some new notions and methods to facilitate the calcu- 
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Fig. 1. The proposed lossless joint source-channel encoding scheme based on linear codes for multiple-access channels in |37 | 



lation of spectra, including properties of vector permutations 
(Section HV-At . a viewpoint of regarding coding modules as 
conditional probability distributions (Section IIV-BI ). spectrum 
generating functions and spectra with coordinate partitions 
(Sections IIV-CI and HV-Dl i. and the MacWilliams identities 
(Section HV-B . 

In Section [V] we review the original requirements of good 
linear codes for lossless source coding, channel coding, and 
lossless JSCC, respectively. To each original requirement there 
just corresponds a spectrum criterion of linear codes, which is 
the kernel-spectrum condition, the image-spectrum condition, 
and the joint-spectrum condition, respectively. Then with each 
criterion there is associated a family of linear codes. They 
are called 6 -asymptotically good linear source codes (LSCs), 
8 -asymptotically good linear channel codes (LCCs), and 5- 
asymptotically good linear source-channel codes (LSCCs), 
respectively. We show that, under some conditions, good 
LSCCs are also good in the sense of LSCs and LCCs. Thus the 
problem of constructing good LSCCs (i.e., linear codes with 
good joint spectra) is of particular interest and importance. 

Based on this observation, in Section I VII we proceed to 
study the general principles for constructing good LSCCs. In 
Section IVI-AI we provide a family of good LSCCs which 
are derived from so-called maximum-rank-distance (MRD) 
codes. In Section [VI-BI we investigate the problem of how to 
construct a good LSCC with the same kernel (resp., image) of 
a given good LSC (resp., LCC). In Section IVl-Cl we propose 
a general serial concatenation scheme for constructing good 
LSCCs. 

In light of this general scheme, in Section lVlIl we turn to the 
analysis of joint spectra of regular low-density generator ma- 
trix (LDGM) codes. We show that the joint spectra of regular 
LDGM codes with appropriate parameters are approximately 
(^-asymptotically good. Based on this fact, we finally present 



a family of good LSCCs by a serial concatenation of an inner 
LDGM code and an outer LDPC code. 

All the proofs are presented in Section [Villi and Section HXl 
concludes the paper. 

II. Notations and Conventions 

In this section, we introduce some basic notations and 
conventions used throughout the paper. 

• In general, symbols, real variables, and deterministic 
mappings are denoted by lowercase letters. Sets, matrices, 
and random elements are denoted by capital letters. 
Alphabets are denoted by script capital letters. 

• The symbol denotes the empty set. 

• The cardinality of a set A is denoted by \A\. 

• The symbols Z, N, No, M, C denote the ring of integers, 
the set of positive integers, the set of nonnegative integers, 
the field of real numbers, and the field of complex 
numbers, respectively. For a prime power q > 1 the 
finite field of order q is denoted by ¥ q . The multiplicative 
subgroup of nonzero elements of ¥ q is denoted by F* 
(and similarly for other fields). 

• For any n € N, we define the index set I n = {1, . . . , n}. 

• The symbol X n denotes the n-fold Cartesian product of 
the alphabet X. A sequence (or vector) in X n is denoted 
by x = x\X2 ■ ■ ■ x n , where Xi denotes the ith component 
of x. For any set A = {ai, a^, ■ ■ ■ , a r } C I n with ai < 
a2 < • • • < a r , we define the sequence (xi)i^A or xa as 
^01^02 ' ' ' x a r - Clearly, one may write x% n in place of 
x. We use the notation a 1 for the I times repetition of a 
single symbol a € X, or simply write a to represent an 
all-a sequence without explicitly indicating the length. 

« The length of a sequence x is denoted by |x|. 

• By default, any vectors are regarded as row vectors. An 
mxn matrix is denoted by M = (Mij)i e x m .;/ez„, where 
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Mi j denotes the (i, j)-th entry. The transpose of a matrix 
M is denoted by M T . The set of all m x n-matrices over a 
field (ring) F is denoted by F mxn . The set F mxn forms 
a vector space (resp., module) over F and in the square 
case m = n an P-algebra relative to the vector space 
(resp., module) operations and matrix multiplication. 
For any functions / : X — > y and g : y — > Z, we define 
the composite g o / : X -> Z by (.9 o = g(f(x)). 

For any functions f : X\ —¥ 3^1 and g : A2 -^3^2, we 
define the Cartesian product / g : X\ x X2 — > 3^1 x ^2 
by (/© 5)(^i,a;2) = (/(a;i),g(x2)). 
The function 1{-} is a mapping defined by l{true} = 1 
and 1 {false} = 0. Then, the indicator function of a subset 
A of a set X can be written as 1{i e i}. 
A permutation on n letters is a one-to-one mapping of 
l n onto l n . The group of all permutations on n letters 
is denoted by S„. 

For a finite set A, the ring of polynomials in the indeter- 
minates X a , a G A, and with coefficients in C is denoted 
by C[X a ; a e A]. The ring C[X a ;a e A] can be defined 
as a subring (consisting of all elements of finite support) 
of the semigroup algebra of N5 4 (the additive semigroup 
of all mappings A — > No) over C. For readers not familiar 
with this concept we remark that for any ordering A = 
{ai, a 2 , . . . , a n }, n = \A\, of the elements of A we obtain 
an isomorphism C[Xi,X2, ■ . ■ , X n ] -4- C[X a ; a e A] by 
setting X l i->- X ai . 

When performing probabilistic analysis, all objects of 
study are related to a basic probability space (fi, ^4, P) 
where ^4 is a cr-algebra (or cr-field) in ft and P is a 
probability measure on (Q,A). For any event A E A, 
PA = P(A) is called the probability of A. Any measur- 
able mapping of ft into some measurable space (B,B) 
is generally called a random element. 
For any random elements F and G in a common mea- 
surable space, the equality F = G means that F and G 
have the same probability distribution. 
For any sequence of real- valued random vari- 

ables, we introduce two probabilistic limit operations, 
i.e., the limit superior in probability p-lim sup,^^ Z n 



defined by 

p-lim sup Z n = inf \ a lim P{Z n > a} = > 

and the limit inferior in probability p-lim inf Z n 
defined by 

p-lim inf Z n = sup (/3 I lim P{Z n < (3} = o\ . 



As 



p-lim sup Z n = p-lim inf Z n = c 
we say that the limit in probability of {Z n }^ =1 is 



p-lim Z„ = c. 

For any random sets or functions, we tacitly assume that 
their n-fold Cartesian products (e.g., A n or Q™ =1 F) are 
Cartesian products of their independent copies. 
All logarithms are taken to the natural base e and denoted 
by ln. 

For any probability distribution P on some alphabet X, 
the entropy H(P) is defined by 



For any probability distributions P and Q on some alpha- 
bet X, the information divergence D(P\\Q) is defined by 

For any number x, y G [0, 1], the information divergence 
function D(x\\y) is defined by 

D(x\\y) = D((x,l-x)\\(y,l-y)). 

For i£l, \x~\ denotes the smallest integer not exceeded 
by x. 

For any real functions f(n) and g(n) with n G N, the 
asymptotic O-notation f(n) = Q(g(n)) means that there 
exist positive constants c\ and C2 such that 

cis(n) < /(n) < c 2 5(n). 

for sufficiently large n. 
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III. Basics of the Code-Spectrum Approach 

In this section, we briefly introduce the basics of the code- 
spectrum approach (37), which is a generalization of the 
weight-distribution approach, e.g., [10], ll35ll . 

Let X and y be two finite (additive) abelian groups. We 
define a linear code as a homomorphism / : X n — > y m . The 
(coding) rate R(f) of / is defined to be the ratio n/m. 

Note that, by defining 

<j(x) = x a -i (1) x a -i(2) ■ ■•2V- 1 (n) Vx G X n 

any permutation a G S„ induces an automorphism of X n . For 
each n G N, we denote by £„ a uniformly distributed random 
permutation on n letters, that is, 



P{£„ =a} = 



1 



Vcr G S„. 



We tacitly assume that different random permutations occur- 
ring in the same expression are independent, and the notations 
such as E m and £„ represent different random permutations 
though it is possible that m = n. 

Next, we introduce the concept of types 0. The type of 
a sequence x in X n is the empirical distribution P x on X 
defined by 

1 1x1 

P x (a) = |-r = «} Va e x - 

' x ' i=i 

For a (probability) distribution P on X, the set of sequences 
of type P in X n is denoted by Tp(X). A distribution P on 
X is called a type of sequences in X n if Tp{X) ^ 0. We 
denote by V(X) the set of all distributions on X, and denote 
by V n {X) the set of all types of sequences in X n . 

Now we introduce the concept of spectra. The spectrum of 
a nonempty set A C X n is the empirical distribution S X (A) 
on V n {X) defined by 

S X {A){P) ± l{x£A 'f x ^ P}l VP e V n {X). 
\ A \ 

The spectrum of A C AT" is closely related to the complete 
weight distribution of A (see for example, |fl9l Ch. 7.7] or ll28l 
Sec. 10]), which can be defined as follows: For x G X n and 
a G X let N x (a) = \{i G I n \ x i — a }\ — n P*.(fl)- This defines 
a function N x : X ^ Nq with X^aeA" -^x(a) = n. The com- 
plete weight distribution of A is then defined as the function 



Nf -> No, TV m. |{x G A|iV x = N}\ = \A\ ■ S x {A){N/n). 
(For the last equality we need to assume that N/n G V n (X), 
i.e. J^aex N(a) — n, but this is only a technical issue.) 
Spectrum and complete weight distribution thus differ only 
by scaling factors in both domain and range. 

Analogously, the joint spectrum of a nonempty set B C 
X n x y m is the empirical distribution S xy {B) on P n {X) x 
V m (y) defined by 

|{(x,y) GB|P x = P,P y =Q}\ 



S xy (B)(P,Q) = 



\B\ 

VP eP n {X),Q eV m {y). 



By considering the marginal and conditional distributions of 
Sxy(B), we obtain the marginal spectra Sx{B), Sy(B) and 
the conditional spectra Sy\x{B), Sx\y(B), that is, 

Sx(B)(P)= Yl Sxy(B)(P,Q) 

Qev m {y) 

Sy(B)(Q)= Yl S X y(B)(P,Q). 



S ylx (B)(Q\P) 



S x \y{B)(P\Q) ± 



S X y(B)(P,Q) 
S X (B){P) 

VP satisfying S X (B)(P) ^ 

S X y(B)(P,Q) 
Sy(B)(Q) 

VQ satisfying Sy(B)(Q) ^ 0. 



Furthermore, for any given function / : X n — > y m , we 
can define its joint spectrum Sxy(f), forward conditional 
spectrum Sy\x(f), backward conditional spectrum Sx\y(f), 
and image spectrum Sy(f) as S X y(rl(f)), Sji|^(rl(/)), 
Sx\y(r\(f)), and Sy(r\(f)), respectively, where rl(/) is the 
relation defined by {(x, /(x))|x G X n }. In this case, the 
forward conditional spectrum is given by 

Sy lX (f)(Q\P) - Sx{xn){p) ■ 

If / is a linear code, we further define its kernel spectrum 
as S x (kerf), where ker/ = {x G X n \f(x) = m }. In 
this case, we also have Sy(f) = Sy(f(X n )) since / is a 
homomorphism of groups. 

The above definitions can be easily extended to more gen- 
eral cases. For example, we may consider the joint spectrum 
Sxyz(C) of a set C C X n x y m x Z\ or consider the 
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MI-2 
u 2 



joint spectrum SxtX^yiyaiff) of a function g : X" 1 x X r 2 

A series of properties about the spectra of codes were given 
in ll37l Sec. II]. We list below some results for easy reference. 

Proposition 3.1: For all P £ V n {X) and P G T ni (X z ) 
(1 < i < m), we have 



S x (X n )(P) 



(np) 



\x\ 



S Xl V - IX - U } : /*, - - - - P m ) = [I Sx, (At ) (Pi) 



where 



nPj ru*(^p(a))! 

and A4 C Xj H (1 < i < m). 

Proposition 3.2: For any given random function F : X n — > 
y m , we have 

P{P(x) =y} = \y\- m a(F)(P Xl P y ) (1) 

for any x G Af™ and y G y m , where 

(2) 



and 



a(F)(P,Q) 



E[Sxy(F)(P,Q)} 



(3) 



SAry(Af»xy^)(P,Q)" 
Remark 3.3: For readers accustomed to thinking rather 
combinatorially than probabilistic, the following alternative 
expression for P{P(x) = y} which is hidden in ([1) and 
(01, would be helpful: because of the symmetrization effect of 
uniform random permutations, P{P(x) = y} depends only 
on the type-pair (P,Q) = (P x ,P y ) of (x, y) and equals the 
expected number of pairs (x', P(x')) ("points of the graph of 
F") of type-pair (P, Q) divided by the total number of pairs 
(x', y') e X n x y m of type-pair (P, Q). For a single encoding 
map / it is just the fraction of pairs (x',y') G X n x y m of 
type-pair (P, Q) which are points of the graph of /. 

Proposition 3.4: For any given linear code / : X n —> y m , 
we have 



*(/)(p ~,p y ) = |^| m i{ y = o m }, v y e y 



(4) 



If both X and y are the finite field F 9 , we define a particular 
random linear code P g R ^ : F^ 1 -4 F™ byx4 xA„ xm , 
where A nxm denotes a random nxm matrix that is uniformly 



distributed over the set F™ x m of all n x m matrices over F g . 
Then we have 

P{P^(x) = y} = P{P 9 R ^(x) = y} = q - m (5) 
for all x e F™\{0"} and y G F™, or equivalently 

< C J(^e) = l (6) 

for all P G P„(F 9 )\{P n} and Q G P„(F 9 ). 

Proposition 3.5: For a given random linear code P : <Y™ — ► 
y m , we have 

p{p(x) = y } = i^r m (7) 

P{P(x) = y|P(x) = y} = |y|- m a(P)(P i _ x , P y _ y ) (8) 
for any unequal x, x G X n and any y,y£ y m , where 

P(x) = P(x) + F m Vx G X n (9) 
and F m is an independent uniform random vector on y m . 

IV. Some New Methods and Results of the 
Code-Spectrum Approach 

In order to evaluate the performance of a linear code, we 
need to calculate or estimate its spectrum. However, the results 
established in [37] are still not enough for this purpose. So in 
this section, we shall present some new notions and methods 
to facilitate the calculation of spectra. For simplicity, some 
definitions and results are stated only for typical cases. Their 
extensions to more general cases are obvious and immediate, 
and hence will be used tacitly in subsequent derivations. 

A. Vector Permutations 

Given an m-tuple cr = (&\, . . . , a m ) G Il"=i S ni < we define 
the vector permutation as the Cartesian product O^ii <J\ of 
permutation-induced automorphisms Ui : X ni — > X Ui , that is, 

Cm \ m 

Qai (x x J = MxO) ieIm Vx Zm G X ni . 
i=l / i=l 

We denote this Cartesian product also by cr. Note that the 
vector permutation is by no means a Cartesian product of per- 
mutations. However, for convenience and also because we are 
only interested in Cartesian products of permutation-induced 
automorphisms, we shall abuse the term "permutation" for 
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permutation-induced automorphisms when there is no possible Clearly, rl(F) = rl(F), and hence Proposition 14.21 yields a 



ambiguity. generalized version of Proposition 13.21 that is 



Define E ni) ... )nm = (S ni) . . . , E„ m ), so that 

1 



P{F(x Z( )=yxJ 

-. m k 

p{s ni ,...,„ m =*}= TfW — v<t E = n i^r m< • a(F)((p^) ieXl ,(p yi ) ieXk ) 



=i i=i 

l k 



We obtain a uniformly distributed random vector permutation. \/-x. x g J~J ^n* y x ^ £ J^J y nlj (15) 

The next two propositions give some general properties of i=1 J =1 

vector permutations. where a(F) = a(rl(F)). 

Proposition 4.1: For any vector permutation er e Jli^i S ni Tne proof of Proposition |4J] is omitted, while the proof of 

and any set A C J]™ x A?"*, Proposition l4~2l is presented in Section IVTlI-AI 

Sx 1 ---x m (c(A)) — Sx 1 —x m (A). (10) ^ Codes and Conditional Probability Distributions 

Proposition 4.2: For any x Xm £ J]™ i <*T and an y random In this subsection ' we introduce one important notion that 

set A C Y[ m i X ni an ^ coc ' e ma y be regarded as a conditional probability distri- 



E 



l{xj m 6 j} 
14 



q(A)((P Xi ) <eIm ) 



bution. Such a viewpoint is very helpful when calculating the 
(11) spectrum of a complex code consisting of many simple codes. 
Proposition 4.4: For any random function F : Yli=i 



k 



where ,,m, 

I = S ni) ..., nm (A) (12) , 

and [ j=i 

pre nv^P ^ M = E \- S yi-yk\x 1 ---x l (F)(Qx k \(P Xi )iex l )} 

i=l J=l 

Moreover, for any nonnegative I < m and any Pj 6 P ni (Ai) wnere 

(/+l<i<m), r^oS tll ...... (17) 



E 



(x2,xnr= 1+ i^(*))ni 



l-4| 



Remark 4.5: The identity ( [T6| > can also be rewritten as 



k 



E[s Xl ... Xm (A)((p Xi ) ieXl ,p Wl )] p{F-^ Xl )^\{r Q n ;{y 



nUi^i n< nU^(^r)( p xj ' ^ i=1 

= S[5 3 ; 1 ... % |^...^(P)(QxJPx I )] 

Remark 4.3: The identity (fTTT > can be rewritten as 

which clearly indicates that the average forward conditional 

\A\ 

P{xx m e A} = ^ m ^ x ^ n . a(A)((P Xi ) ie x m ) spectmmE[S yi ...y k \ Xl ... Xl {F)(Q Xk \P X[ )} may be regarded as 

the transition probability from P Xl to Q Xk under P~. This 



whenever |A| is a constant. For a given random function F : 
X n _^ -ym^ let ^ = rl ( F ) and note that |^| = |^|n then 

it is obvious that Proposition 13.21 is an easy consequence of 
Proposition 14.21 Generally, for a given random function F : 
nL <*T nti 3^ we can define 



fundamental observation implies that coding modules like F~ 
or F (instead of F) should be regarded as basic units in 
a coding system, and that the serial concatenation of such 
units may behave like the serial concatenation of conditional 
probability distributions. The following proposition proves this 



F = S TO1> ... >mk o F o S ni) ... , ni . speculation. 
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Proposition 4.6: For any two random functions F : X n — > 

y m and G : y rn -> Z\ 

E[S zlx (GoE m oF)(Q\0)} 
= E[S zlx (G~ o F~)(Q\0)} 
= E E[Sy lx (P\0)]E[S zly (Q\P)] 

VO eP n (X),Q ePi(Z). (18) 
The proofs of this subsection are given in Section IVIII-BI 

C. Spectrum Generating Functions 

The spectrum generating function of a set A C X n is a 
polynomial in | X\ indeterminates, whose coefficients form the 
spectrum of A. As an element of C[u a ;a <E X] it can be 
succinctly defined as 

1 - 

g x {A)(u)± m £IK 

= j2 f^(^)(^n< p(a) ) 

P£V n (X) \ aGX ) 

where u = (u a ) a€X . For convenience, we shall write u nP in 
place of JlaGA" u a P ^ aS> in the sequel. 

We can associate similar generating functions with other 
types of spectra defined in Section [TTTJ the most important 
of which is the joint-spectrum generating function of a set 
B C X" x y m ; 

Gxy(B)(u, v) 

= £ ^(B)(^Q)-u" P v m « 

Qe-P m (y) 

1 1 (x,y)es 

a polynomial in the \X\ + \y\ indeterminates u a , a £ X, and 
Vb, bey, where u = (u a ) aeX and v = (v b ) bey . 

To simplify notations, we sometimes write Q(A)(u) and 
Q(B)(u, v) when there is no possible ambiguity. 

When the set A C is random, its associated spectrum 
generating function is also random. To analyze a random 
polynomial, we need to consider its expectation. However, the 
expectation of a random polynomial is indeed undefined in 
the usual sense of expectations, so it is necessary to give its 
definition formally. Let C[u a ; a G A] be a ring of polynomials, 



where A is a nonempty finite set. Since the ring is a subset of 
C N o* , namely, all mappings of Nq into C, it can be endowed 
with the er-algebra S = C[u a :a e A] H B N » where B is 
the Borel a- algebra of C. For a given random polynomial 
P(u) : fi — > C[u a ; a £ A], its expectation is defined as 

E[P(u)]= £[coef(P(u),u n )] -u n 

where coef (p(u), u n ) denotes the coefficient of the mono- 
mial u n in p(u). To see that it is well defined, note that 
coef (p(u), u n ) with n fixed is a canonical projection from 
C N " to C and hence measurable. Analogous to ordinary 
expectations, expectations of random polynomials have the 
following properties: for any random polynomials Pi(u) and 
^ 2 (u), 

B[Pi(u) + P 2 (u)] = P[Pi(u)] + P[P 2 (u)]; 

if Pi(u) and P 2 (u) are independent then 

£[P x (u)iMu)] =£[Pi(u)]£[P 2 (u)]. 

We leave the proofs to the reader. The next proposition gives 
an important property of expectations of random spectrum 
generating functions. 

Proposition 4.7: Let F : X -)• X and G : y -4 y be two 
independent random bijective mappings. Then for any random 
setP C X n xy m , the expectation E[G((F n QG m )(B))(u, v)] 
can be obtained from E[Q(B)(u, v)] by the substitutions 

u a i V E[u F[a) ] VaeX (19) 

and 

v b i— > B[«g(«] V6 G y (20) 

where P„ = 0"=i ^ and G ™ = OJLi G - 
The proof is given in Section IVIII-CI 

D. Spectra with Coordinate Partitions 

In Section IIV-BI we introduced the method for calculating 
the spectra of serial concatenations of codes. In this subsection, 
we proceed to investigate the other important kind of combina- 
tions of codes, namely, parallel concatenations. To cope with 
the problem involving concatenations (products) of sequences, 
we first need to introduce a generalized definition of spectra, 
namely, spectra of sets with coordinate partitions. 
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Let A be a subset of X n , with coordinate set X n . Let U be 
a fixed partition of I n , namely, a set of nonempty subsets of 
I n such that every number in I n is in exactly one of these 
subsets. We define the U-type of a sequence x € X n as 

(Px)ueu = {Pxu)ueu- 

By Vn(X) we mean the set of all iY-types of sequences in 
X n , so that 

v u n {x)= Y[r m (x). 

ueu 

Any W-type in Vn(X) is denoted in the form P w = (P u ) UeU . 
Based on the W-type, we define the U-spectrum of A as the 
empirical distribution S x u{A) on V^(X) given by 



|{xeA|P x " = P"}| 
14 



VP" G T%{X). 



The W-spectrum is just a variant of the joint spectrum. When 
U = {I n }, it reduces to the ordinary spectrum. 

Analogously, given a set B C A*" x y m and a pair of 
coordinate partitions U of I„ and V of Z m , we can define 
the joint (U, V)-spectrum S x u yV (B) on P^(^) x pVQ>). 
Furthermore, given the function / : X" — > y m with a pair 
of coordinate partitions (U, V), we define its joint (U, V)- 
spectrum S x uyv(f) as the joint (W, V)-spectrum of its as- 
sociated relation rl(/). Similar generalizations follow easily 
for conditional and marginal spectra. 

The above concepts are crucial for computing the spectra 
of parallel concatenations of codes, but the notations are 
somewhat complex. For convenience, when we explicitly write 
A C YIa=i X th with Y%=i n i ~ n > we tacitly assume that the 
default coordinate partition is 

U = {U U U 2 ,--- ,U P } 

= |{l,-..,n 1 },.-.,|^n l + l,..-,n||. (21) 

Thus the default spectrum of A is the Z//o- s P ec trum of A 
and is denoted by S X ... X (A) or S X p(A). When p = 1, this 
convention just coincides with the ordinary spectrum of A. 

Now, even for a single set A C X n , there are many different 
spectra of A with respect to different coordinate partitions. So 
next, let us investigate the properties of spectra of sets with 
coordinate partitions. For this purpose, we need to generalize 
the spectrum generating functions introduced in Section ITV-CI 



For any set A C X n with a coordinate partition U, we 
define its U-spectrum generating function Qx u (A) as 

g x u(A)(uu) = rn X! II u tf IF * 

1 1 xeAUGU 

= £ S x u(A)(P«)l[uW PU 

pUtz V U(x) U&A 

a polynomial in the indeterminates Uu,a> U £ 14, 

a e X, where u u = (uu) UeU = {u u<a ) Ue u,azx- When 
U = {!„}, this definition reduces to the ordinary spectrum 
generating function Qx(A)(u). 

Analogously, given a set B C X n x y m with coordinate 
partitions U and V, we can define the (14, V)-spectrum gener- 
ating function Gx u y v (B)(uu, vy). And furthermore, given a 
function / : X n — > y m with coordinate partitions U and V, 
we define its (U, V)-spectrum generating function by 

Gx"yv(f)(u u ,v v ) = g x u y v(r\(f))(u u ,v v ). 

Again for convenience, when we explicitly write A C 

n<=i w i tn 2f=i n i = n > we tacitly assume that the 
default coordinate partition is Uq defined by (|2TT >. Thus the de- 
fault spectrum generating function of A is its £Y - s P ec trum gen- 
erating function which we denote by g x ... x (A)(u\, ■ ■ ■ ,u p ) 
or Qxp(A)(uz ). When p = 1, it coincides with the notation 
Sx(A)(u). 

Based on the above definitions, we proved the following 
properties: 

Proposition 4.8: For any sets A C X" and B C y m , 



g(A x p)(u,v) - g(A){u) ■ g(B)(v). 



(22) 



Proposition 4.9: Suppose Ui and W2 are two partitions of 
I n . If U2 is a refinement of U\ (which means each element of 
U\ is a union of elements of U2), then g x u 1 (A)(ui t u x ) can 
be obtained from g x u 2 (A)(vL2^Ui) d Y the substitution 



u 2 ,c/ 2 i-> Ul,^ 



if t/ 2 C Ui 



(23) 



where [/1 <E Wi and L/2 6^/2- 

By Propositions 14.81 and 14.91 we easily obtain a series of 
corollaries as follows. 

Corollary 4.10: For any two sets A\ C X ni and A 2 C 
X n2 , we have 

^(Ax x A 2 )(u) = ^(Ai)(u) • ^(A 2 )(u). 
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Corollary 4.11: 

Q(X n )(u) = [Q{X){n)r = 



A 2iriTr(x)/p 



E\ n 



\x\ 



Tr(x) = x + x p + ■ ■ ■ + x p 



(26) 
(27) 



Corollary 4.12: For any two maps /j : X ni — > y mi and 
/ 2 : X n2 y m \ we have 

Gxy(fi /a)(u,v) = Gxy(fi)(u, v) • G X y(f 2 )(u, v). 



The proofs of Corollary 14. Ill are omitted, while the proofs 
of the other results are given in Section IVIII-DI Note that 
Corollary gTU is an easy consequence of Corollary 14.101 and 
that Corollary 14. 121 is just the tool that we want for computing 
the spectra of parallel concatenations of codes. 

E. MacWilliams Identities 

One of the most famous results in coding theory is the 
MacWilliams identities [23], which relate the weight enumer- 
ator of a linear code to that of its dual code. In this subsection, 
we shall generalize this result in the framework of code- 
spectrum approach. This generalization may be regarded as 
a combination and extension of the results in ifTBI . 11331 . 1351 . 

To make things easier to understand, we shall mainly 
consider the special case that the alphabet is a finite field ¥ q 
with q = p r , where p is prime. 

For any xi, x 2 <E F™, we define the standard dot product on 
F»by 

n 

Xi • X 2 = y^X M X2,i. 

i=l 

Then for any set A C F^', the orthogonal set A 1 - of A is given 
by 

A 1 - = {x G F';|Vx e A :x-x =0}. 

Clearly, A 1 - is a subspace of F™. 

The next theorem gives the generalized MacWilliams iden- 
tities in the framework of code-spectrum approach. 

Theorem 4.13: Let A be a subspace of F^ 1 and U a fixed 
partition of I n . Then 

1 



g ¥ u{A^){n u ) 



-g v u{A){{uuM) um ) (24) 



where M is the q x q matrix (indexed by the elements of F 9 ) 
defined by 



Remark 4.14: Note that Ti(x) is an F p -module epimor- 
phism from ¥ q onto F p , and hence xi x ) i s a homomorphism 
from the additive group of V q to C x . Moreover, it is easy to 
see that 

2 X (ax) - ^ X (x) =0 Va ^ 0. (28) 

xeF q xew,, 

From this fact, it follows that q~^M is a symmetric, unitary 
matrix. In particular, 



53 M a , x = 53 M x,a = ql{a = 0}. 



(29) 



x£¥„ 



One important application of Theorem 14.131 is calculating 
the joint spectrum of a linear code x = yA T when the joint 
spectrum of y = xA is known. The next theorem gives the 
details. 

Theorem 4.15: Let A be an n x m matrix over ¥ q . Define 
the linear codes / : F™ — > F™ and g : F™ -> F™ by 

/(x) - xA Vx G F' 9 l (30) 

and 

. 9 (y)=yA T VyeF™ (31) 

respectively. Let U be a partition of I n and V a partition of 
Xm. Then 



g F v ¥ u(~g)(v Vl u u ) = 

■^Q*«Mf)((uvM)u&t, (vyM) yeV ) (32) 



where M is the matrix defined by 

The above results can be further generalized by replacing 
¥ q and \ w ith a Frobenius ring and its generating character, 
respectively. A ring X is said to be a Frobenius ring if there 
exists a group homomorphism \ '■ {X,+) —> C x (character 
of (X, +)), whose kernel contains no nonzero ideal. Such a 
homomorphism is called a generating character of X. For 
background information on Frobenius rings and their uses in 
coding theory we refer to ATI. I12T1 §161. Il26l and ll35l. 

Theorem 4.16 (cf. also [Wj): Let X be a Frobenius ring, 
A a submodule of X n , and U a fixed partition of I n . We have 
1 



M„ 



x(aia 2 ) Vai,a 2 £ ¥ q 



(25) 



g x u(A±)(uu) 



-G x u{A){{xiuM) UeU ) (33) 
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where M is the \X\ x | X | matrix (indexed by the elements of 
X) defined by 

M 01i02 = x(aia 2 ) Vai,a 2 G X (34) 

for some generating character \ of X. 

The proofs of this subsection are given in Section IVIII-EI 

V. Good Linear Codes for Source Coding, 
Channel Coding, and Joint Source-Channel 
Coding 

In the world of classical coding theory, a good linear code 
always means a set of codewords that has good performance 
for some family of channels or has some good distance 
properties (e.g., the Singleton bound, the Gilbert- Varshamov 
bound, and so on). Such a viewpoint may be enough for 
channel coding, but may have its limitations in the context of 
source coding and JSCC. This is mainly because the approach 
only regarding linear codes as a set of codewords cannot cover 
all the properties of linear codes, and in different applications 
there are different criteria for good linear codes. 

In ||37l Table I], we only briefly reviewed the criteria of 
good linear codes in terms of spectrum requirements for 
lossless source coding, channel coding, and lossless JSCC. 
So in this section, we shall resume this discussion, including 
the concepts of good linear codes and the relations among 
different kinds of good linear codes. 

At first, let us introduce some concepts about rates of codes. 

For any linear code / : X n — > y m , we define the source 
transmission rate R s (f) (nats per source symbol) of / by 

R s (f) = -\n\f(X n )\. (35) 
n 

Analogously, we define the channel transmission rate R c {f) 
(nats per channel symbol) of / by 

R c (f) = -\n\f(X n )\. (36) 
m 

Recall that in Section [HI] we define the (coding) rate R(f) of 
a linear code / by n/m. Then there is a simple relation among 
R.U)> R df\ and R(f), that is, R(f)R s (f) = R c {f). If / is 
injective, then R s (f) = In \X\ and R c (f) = In \X\R(f); if / 
is surjective, then R s (f) = In \y\/R{f) and R c (f) = In |^|. 
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Let F = {F k : X" k — > y mk } k x L 1 be a sequence of random 
linear codes. We define the asymptotic supremum source trans- 
mission rate R s (_F) and the asymptotic infimum source trans- 
mission rate R S (F) by R S (F) = p-lim sup^^ R s (F k ) and 
R S (F) = p-liminffe^.^ R s (F k ), respectively. As R S (F) = 
R S (F), we say that the asymptotic source transmission rate 
of F is 

R S (F) =p-limi? s (F fc ). (37) 

k— >oo 

Analogously, we define the asymptotic supremum channel 
transmission rate R C (F) and the asymptotic infimum channel 
transmission rate R C (F) by R C (F) = p-limsupj,^,^ R c (F k ) 
and R C (F) = p-\im ini k ^ oc R c (Fk), respectively. As 
R C {F) = R C (F), we say that the asymptotic channel trans- 
mission rate of F is 

R c {F)±p-KmR c (F k ). (38) 

k— >oo 

Also, we define the asymptotic supremum coding rate R(F) 
and the asymptotic infimum coding rate R(F) by R{F) = 
limsup^^ R(F k ) and R{F) = liminffc^oo R(F k ), respec- 
tively. As R(F) = R(F), we say that the asymptotic coding 
rate of F is 

R(F) = lim R(F k ). (39) 

k— >oo 

To simplify notations, in the rest of this section, when 
writing F, we always mean a sequence {F k } k x L 1 of random 
linear codes Fk : X rLk — > y nik . To avoid some singular cases, 
we assume in the sequel that for any sequence F of random 
linear codes, lim / t_ s . 00 \Fk(X nk )\ = oo. 

Next, for better understanding of the definitions of good 
linear codes, let us review the original requirements of good 
linear codes for lossless source coding, channel coding, and 
lossless JSCC, respectivelyQ 

According to 11391 , a sequence F of random linear codes 
with the asymptotic source transmission rate R S {F) is said to 
be 5-asymptotically good for lossless source coding if for any 
e > there exits a sequence of events A k G A such that for 
sufficiently large k, 

P(A k ) > 1 - e (40) 

'This review is merely based on the ideas and results in previous papers. 
For technical reasons, the requirements we give here are not the same as those 
in the literature. 
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\R s (F k ) - R S (F)\ < e Vwe4 



(41) 



1 



max _ lnP{F fc (x) = F k {ic)\A k } < -R S (F) + S + e. 

x.x:x/x Tlfc 

(42) 

Note that 



lim P{\R s (F k ) - R S (F)\ <e} = l 

fc— >oo 

so this definition is reasonable. Due to the property of linear 
codes, the condition (l42l is equivalent to 

max — lnP{x e ker F k \A k } < -R S (F) + 5 + e. (43) 

x:x^0"fc n k 

According to OJ, fl3l . OT1 . a sequence F of random linear 
codes with the asymptotic channel transmission rate R C (F) is 
said to be (^-asymptotically good for channel coding if for any 
e > there exits a sequence of events A k € A such that for 
sufficiently large k, 



P(A k ) > 1-6 

\R c {F k ) - R C (F)\ <e Vwe4 

max ±l J P{yeCr yeC Fk \A*} 
y,y:y#y m fc \P{y e C Fk \A k }P{y e C Ffc |A fe } 



(44) 

(45) 

< 5 + e 
(46) 



where C Fk = F k {X nk ) + Y mk and Y mk is an independent 
uniform random vector on y mk . Clearly, 
1 



m k 



lnP{y G C Ffc + In \y\ - R C (F) 



< e 



for all y G y mn . Because F k is a linear code, we also have 

P{yeC Fk ,yeC Fk \A k } 

= P{yeC Fk ,y-yeF k (X nk )\A k } 

= E P i^ = h\Ak}E[l{y eC fk ,y-ye f k (X n «)}} 

= E p ^ = h\M}P{y G C /fc }l{y -ye 

,/ A' 

This together with (l45t gives 

1 P{y £C ft ,y £C ffc |A fc } 



In |^| -J? C (F) 



m k P{y-y€F k {X" k )\A k } 
Then the condition d46b can be rewritten as 

max — \nP{yeF k (X nk )\A k } 

y:y^0 m fc m k 

<-\n\y\+R c (F)+S + e 
by noting that e is arbitrary. 



< e. 



(47) 



According to 071 , a sequence F of random linear codes 
is said to be (5-asymptotically good for lossless JSCC if it 
satisfies 



lim sup max 

k — >oo x,x:x^x 

-1 m r^ (x) ^ y ^^r^l ) < f . (48) 



n k \P{F Fk (x) - y}P{.P Ffc (x) = y} , 
where J> t (x) = F fe (x) + F mfc and Y mk is an independent 
uniform random vector on y nik . By the arguments in the proof 
(in 071 ) of Proposition |3.5l we obtain an alternative condition: 

limsup max — In {\y\ mk P{F k {x) = y}) < 5. (49) 
k ^OD x:x^o n k 

y 

The requirements given above are fundamental, but are not 
easy and convenient for use. The next three propositions show 
that spectra of linear codes can serve as alternative criteria of 
good linear codes, and that the uniform random permutation 
is a useful tool for constructing good linear codes. 

Proposition 5.1: Let F be a sequence of random linear 
codes with the asymptotic source transmission rate R S (F). 
If it satisfies the kernel-spectrum condition: 

limsup max — InaCker F k )(P) < S (50) 

fc^oc PeVn k (X)\{P n k } n k 

then the sequence of random linear codes F k = F k o E„ fc is 
(5-asymptotically good for lossless source coding. 

Proposition 5.2: Let F be a sequence of random linear 
codes with the asymptotic channel transmission rate R C (F). 
If it satisfies the image-spectrum condition: 

limsup max — kia(F k (X nk ))(Q) < S (51) 
fc^oo Qev mk (.y)\{p m k } m k 

then the sequence of linear codes E mfc oF k is (5-asymptotically 

good for channel coding. 

Proposition 5.3: Let F be a sequence of random linear 

codes. If it satisfies the joint-spectrum condition: 

limsup max — In a(F k )(P, Q) < 5 (52) 

Qev mk (y) 

then the sequence of linear codes F k = S„ lf . o F k o £ nj . is 
(5-asymptotically good for lossless JSCC. 
For convenience, we define the function 

p(F)= max -lna(P)(P,Q) (53) 

PeV n (x)\{P n}, n 
Qev m (y) 
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for any random linear code F : X n — > y m . Then the condition 
(|52T > can be written as limsup^^ p(Fk) < 5. 

As shown above, as long as the linear codes satisfy the 
code-spectrum requirements, we can then construct good 
ensembles of random linear codes by random permutations. 
So from now on, we shall use the conditions d50l l. ( BTt . 
and d52l as the formal definitions of <5-asymptotically good 
(random) linear codes for lossless source coding, channel 
coding, and lossless JSCC, respectively. These linear codes are 
then called <5-asymptotically good linear source codes (LSCs), 
5-asymptotically good linear channel codes (LCCs), and <5- 
asymptotically good linear source-channel codes (LSCCs), 
respectively. Alternatively, they are said to be <5-asymptotically 
SC-good, (5-asymptotically CC-good, and <5-asymptotically 
SCC-good, respectively. When 8 = 0, we use the sim- 
plified term "asymptotically good"; when talking about 8- 
asymptotically good LSCs (resp., LCCs), we tacitly assume 
that their source (resp., channel) transmission rates converge 
in probability. Since the conditions of (5-asymptotically good 
LSCs or LCCs are only concerned with the kernel or image 
of linear codes, we also introduce the concepts of equiva- 
lence in the sense of LSCs or LCCs. For any linear codes 
/ : X n -4 y rn and g : X" -> y m , they are said to be SC- 
equivalent if they have the same kernel, and they are said to 
be CC-equivalent if they have the same image. 

The following propositions investigate the relations among 
(5-asymptotically good LSCs, LCCs, and LSCCs. 

Proposition 5.4: For a sequence F of (5-asymptotically 
good LSCs, there exists a sequence G = {Gk}%% 1 of 8- 
asymptotically good LCCs Gk : X lh — > X' ,lk such that 
G k (X l ») = keiF k for all k 6 K 

Proposition 5.5: For a sequence F of (5-asymptotically 
good LCCs, there exists a sequence G = {Gk}^-i of <5- 
asymptotically good LSCs Gk : y mk y lk such that 
kcrGfe = F k {X nk ) for all k E K 

Proposition 5.6: Let F be a sequence of (5-asymptotically 
good LSCCs. Then it is 5-asymptotically SC-good whenever 
its source transmission rate converges in probability, and 
it is (5i?(.F)-asymptoticairy CC-good whenever its channel 
transmission rate converges in probability. 



By Propositions 15.41 15.51 and 15.61 we thus establish the 
relations among different kinds of good linear codes. These 
relations are all depicted in Fig. [2] Among these relations, the 

good LSCCs 

S \ 
good LSCs <^=^ good LCCs 

Fig. 2. The relations among different kinds of good linear codes 

relations (Proposition 15.61 ) that good LSCCs are good LSCs 
and LCCs deserve much more attention. This fact indicates 
the fundamental role of good LSCCs, and allows us to only 
concentrate on one problem, namely, the construction of linear 
codes with good joint spectra. So in the following sections, 
we shall investigate this problem in depth. Also note that, 
however, there are two arrows missing in Fig. [2] one from LSC 
to LSCC and the other from LCC to LSCC. Then naturally, we 
have such a question: are (5-asymptotically good LSCs (resp., 
LCCs) (5-asymptotically SCC-good, or can we construct 8- 
asymptotically good LSCCs which are SC-equivalent (resp., 
CC-equivalent) to given (5-asymptotically good LSCs (resp., 
LCCs)? 

As 8 is close to zero, it is clear that (5-asymptotically good 
LSCs (or LCCs) may not be (5-asymptotically SCC-good. 
11391 Theorem 4] shows that there exist asymptotically good 
LSCs in the ensemble of low-density parity-check matrices. 
However, [37, Corollary 4.2] proves that these codes are not 
asymptotically SCC-good because the matrices are sparse. It 
is well known that, for any linear code over a finite field, 
there exits a CC-equivalent systematic linear code. According 
to 11371 Corollary 4.1], high rate systematic linear codes are 
not asymptotically SCC-good. So we can always find asymp- 
totically good systematic LCCs which are not asymptotically 
SCC-good. 

Now, the remaining question is if and how we can construct 
(5-asymptotically good LSCCs which are SC-equivalent (resp., 
CC-equivalent) to given (5-asymptotically good LSCs (resp., 
LCCs). The answer is positive and will be given in Section IvTl 

We close this section with a simple proposition. 

Proposition 5.7: Let {Fi = {Fi,k}kLi}i^i be a family of 
sequences of random linear codes i 7 ^ : X rLk — > y mk . The 
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sequence Fi (i G N) is ^-asymptotically SCC-good where Si 
is nonincreasing in i and converges to S as i — > oo. Define the 
random linear code Gi tk = Fi a , k where 

i = arg min ,p{F j>k ). 

Then {Gk,k}kLi is (^-asymptotically SCC-good. 

The proofs of this section are given in Section IVIII-FI 

VI. General Principles for Constructing Good 
Linear Source-Channel Codes 

A. A Class of SCC-Good Codes Derived from Certain 
Maximum-Rank-Distance Codes 

Recall that in ||37l Sec. Ill], a random linear code F : X n — > 
y m is said to be good for JSCC if it satisfies 



a(F)(P,Q) = l 



(54) 



for all P G P n (X)\{P on } and all Q G P m {y). To distinguish 
good linear codes for JSCC from good linear codes in other 
contexts, we say that F is SCC-good. It is clear that SCC-good 
linear codes are asymptotically SCC-good. By Proposition l3.2l 
we also have an alterative condition: 

P{F(x) = y} = \y\- m Vxenrj.VyeF. (55) 

The random linear code F^^ (defined in Proposition 13. 4b 
is obviously SCC-good (cf. ©, ©) but in some sense trivial, 
since its distribution has support F™ x ™\ the set of all linear 
codes / : F™ —> F™. In this subsection we provide further 
examples of SCC-good random linear codes with support size 
much smaller than |F™ xm | = q mn . These are derived from 
so-called maximum-rank-distance (MRD) codes, and have the 
stronger property 

P{F(x)=y} = \y\- m Vxer\{0"},Vyer (56) 

which one might call "SCC-good before symmetrization". 

Next we give a brief review of MRD codes. Let n,m,k 
be positive integers with k < min{n,m}. Write n' = 
max{n,m}, to' = niin{rt, m} (so that (n',m') equals ei- 
ther (n, to) or (to, n)). An (n, to, k) maximum-rank-distance 
(MRD) code over ¥ q is a set C of q kn matrices in F" xm such 
that 

rank(A - B) > m' - k + 1 for all distinct A, B G C. (57) 



MRD codes are optimal for the rank distance cIr : (A, B) i— >■ 
rank(A — B) on F^ xm in the same way as maximum distance 
separable (MDS) codes are optimal for the Hamming distance, 
as the following reasoning shows: Let C C F™ xm have 
minimum rank distance d, 1 < d < to'. We may view C 
as a code of length m over F^ (the columns of A G F™ xm 
being the "entries" of A). As such C has Hamming distance 
> d (since a matrix of rank d must have at least d nonzero 
columns). Hence \C\ < g"( m - d + 1 ) by the Singleton bound. 
By transposing we also have \C\ < g m ("- d + 1 ) i so that 



\C\ < min q ™(r> 



d+1 )j = q n '( m '- d + 1 )^ (58) 



is sharp are exactly the (n, to, k) 
1 d+1. 



The codes for which 
MRD codes with k = m' 

MRD codes were introduced in [9 1 under the name "Single- 
ton system" and investigated further in ifTTl . QUI . As shown 
in (9), O], linear (n, to, k) MRD codes over ¥ q (i.e. MRD 
codes which are F 9 -subspaces of F" xm ) exist for all n, m, k 
with 1 < k < min{n,TO,}. The standard construction uses q- 
analogues of Reed-Solomon codes, which are defined as fol- 
lows: Assuming n > to for a moment, let x%, . . . , x m G F g n 
be linearly independent over ¥ q . For 1 < k < to let D k be 
the linear [to, k] code over F g n having generator matrix 



xi 



X-2 



\ 



(59) 



Replace each codeword c = (ci, . . . , c m ) G D k by the n x m 
matrix C having as columns the coordinate vectors of cj with 
respect to some fixed basis of F 9 « over ¥ q . The set C k of 
q nk = \D k \ matrices C obtained in this way forms a linear 
(n, m, k) MRD code over ¥ q . The restriction n > m is not 
essential, since transposing each matrix of an (n, to, k) MRD 
code yields an (to, n, k) MRD code. We shall follow [32| and 
call the codes C k C ¥ q xm (1 < k < to < n), as well as 
their transposes, Gabidulin codes. Thus linear (n, to, k) MRD 
codes over ¥ q exist for all positive integers n, m, k with k < 
min{n, to}. 

For our construction of SCC-good random linear codes we 
need the following property of Gabidulin codes. 
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Proposition 6.1: Suppose C C F™ xm i s a Gabidulin code. 
Then {xA; A e C} = F™ for every x € F£ \ {0"}, and 
similarly {Ay T ; A eC}=FJ for every y G F™ \ {0 m }. 

Theorem 6.2: Let C C F" xm be a Gabidulin code. Define 
a random linear code F : W q L — > F™ by 

ifAeC, 

F{F = A} = ^ |c| 

otherwise. 

Then F is SCC-good. 

Theorem 16.21 provides us with SCC-good random linear 
codes F : F™ — >• F™ of support size as small as q n = 
g,max{n,m}^ rea jj zec [ by an ( nj m ^ MRD code, which has 

minimum rank distance d = m' = min{n, m} (the full- 
rank case). In fact, we showed in [36] that g max {"> m } i s 
the minimum support size of an SCC-good random linear 
code satisfying the stronger condition d56l ), and that a random 
linear code of support size g max {™<« 1 } j s SCC-good before 
symmetrization if and only if it is uniformly distributed over 
an (n,m, 1) MRD code. 

As an application of Theorem 16.21 the next example gives 
us some insights into the general form of a random linear code 
that is SCC-good before symmetrization. 

Example 6.3: Let m = n = q = 2 and F4 = Fa [a] with 
a 2 = a+1. It is clear that {1, a} is a basis of F4. By definition 
of Gabidulin codes, we may consider the generator matrix 
Gi = (1 a). Then the Gabidulin code C\ is 

{(0 0),(1 a), (a l + a),(l + a 1)} 

or 

'0 0\ / 1 0\ /0 l\ / 1 r 

v° °/A° V A 1 VV °y 

Note that C\ is isomorphic to F4. By Theorem l6.2l the random 
linear code uniformly distributed over C\, which we also 
denote by C±, is SCC-good. Interchanging the columns of each 
matrix in C\ gives another SCC-good random linear code 

'If '),(' \(** 

v o OJ \l 0/ \0 1/ \1 1 

The cosets of C\, such as 

i\ (1 i\ /o o\ /i 0' 



c 2 



are also SCC-good. Note that C3 contains only one matrix of 
full rank. Finally, let C = Cj, where / is an arbitrary random 
variable taking values in Z3. It is clear that C is SCC-good. 
The proofs of this subsection are given in Section IVIII-GI 

B. Constructing Good LSCCs Based on Good LSCs or LCCs 

The condition d55l ) is a very strong condition, which in fact 
reflects the property of the alphabet. Therefore, combined with 
the injective property of mappings, we say that an alphabet 
(an abelian group) X is super good if there exists a sequence 
{F n }^ =1 of independent SCC-good random linear codes F n : 
X n -» X n such that 



inf P{|kerF„ 

new 



1} > 0. 



(60) 



We denote by {-F^„}^Lx tne sec l uence of such SCC-good 
random linear codes. Then, a natural question is what abelian 
groups are super good. 

Recall that X is said to be elementary abelian if X is 
isomorphic to 1 S V for some prime p and some integer s > 0. 
The nontrivial elementary abelian p-groups are exactly the 
additive groups of the finite fields ¥ q with q = p r . 

From Proposition 13.41 and a well-known result (Theo- 
rem l6.41 i. it follows that all elementary abelian groups are super 
good. 

Theorem 6.4 (see iZK)-' Let rank(J) denote the rank of the 

Then 



generator matrix of the linear code / : F™ — > F£ 

n 

P {rank(F^) = n ) = JJ (1 _ g -i) > Kq 

i=l 



(61) 



where K q = Yl°Z 1 (l — q l ) can be expanded as an alternating 



Ec- 1 ) 



c, = 



0/ \0 1/ \1 1/ \1 0, 



fc=l 

by Euler's pentagonal number theorem (see (2)). 

The next theorem shows that, conversely, a super good finite 
abelian group is necessarily elementary abelian. 

Theorem 6.5: For nontrivial abelian groups X and y, if 
there exists an SCC-good random linear code F : X n —> y m , 
then X and y are elementary abelian p-groups for the same 
prime p (provided that m, n > 1). 

Now let us investigate the relation between conditions (l53t 
and ( f60b for elementary abelian groups. 
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Theorem 6.6: Let X be an elementary abelian group of 
order q = p r . Then for every SCC-good random linear code 
F : X n -> X n the following bound holds: 

P{\kerF\ = l}> P ~ 2 + q ~ n . 

P - 1 

From Theorem 16.61 an immediate consequence follows. 

Corollary 6.7: Let X be an elementary abelian p-group for 
some prime p > 2. Then there exists a positive constant 
c(|A?|) (which may be taken as c(|A"|) = 1 — jzj) such that 
P{\ kcrF| = 1} > c( | A" | ) for every SCC-good random linear 
code F : X n -> X n . 

However, the conclusion of Corollary 16.71 does not hold for 
p = 2. 

Proposition 6.8: If X is an elementary abelian 2-group, 
there exists a sequence {F n }^ =1 of SCC-good random linear 
codes F n : X n -> X n such that lim„^oo P{\ kerF n | = 1} = 
0. 

With the preparation above, let us investigate the problem 
of how to construct (5-asymptotically good LSCCs which are 
SC-equivalent (resp., CC-equivalent) to given <5-asymptotically 
good LSCs (resp., LCCs). 

The next theorem gives some ways for constructing SC- 
equivalent or CC-equivalent linear codes. 

Theorem 6.9: Let F : X n — > y m be a random linear code. 
If y is an elementary abelian group of size q, then the random 
linear- code G x = F^ c m oF = F^ c m o S m o F satisfies 

P{ker G x = ker F} > K q (62) 
E[Sy\ x {G x )<Q\P)\ 

< Sy(y m )(Q) + HQ = P ™}E[S y]x (F)(P om \P)] 

ypev n (x)\{p n},Qev m (y). (63) 

If X is an elementary abelian group of size q, then the random 
linear code G 2 = F o F*£° = F o E„ o F^c satisfies 

P{G 2 {X n ) = F(X n )} > K q (64) 
E[S ylx (G 2 )(Q\P)} = E[S y (F(X n ))(Q)] 

VPeV n (X)\{P n},Qev m (y). (65) 

Based on Theorem 16.91 we thus find the answer of the 
problem by the following theorem. 



Theorem 6.10: Let / = {fk}fcLi be a sequence of linear 
codes fk : X ,lk — >■ y nik . If / is a sequence of <5-asymptotically 
good LSCs satisfying 

R c (f)=\n\y\ (66) 

and y is elementary abelian, then there exists a sequence 
{ffi,fc}fe°=i °f (^-asymptotically good LSCCs g x ,k ■ X nh -> 
ym k g^jj (jj a j g i k j s SC-equivalent to fk for each k £ N. 
Analogously, if / is a sequence of (5-asymptotically good 
injective LCCs and X is elementary abelian, then there exists a 
sequence {g2,k}kLi °f <V-R(/)-asymptotically good injective 
LSCCs 52, k ■ X n " -> ^ mfc such that g 2y k is CC-equivalent to 
fk for each fc £ N. 

Theorem 16.101 is a very fundamental result, which not 
only claims the existence of SC-equivalent (or CC-equivalent) 
(5-asymptotically good LSCCs but also gives the way for 
constructing such good LSCCs by concatenating rate-1 linear 
codes. Recalling that rate-1 linear codes (e.g., the "accumu- 
late" code) are frequently used to construct good LCCs (e.g., 
CD, GO), ETJ), we believe that finding good rate-1 LSCCs is 
an issue deserving further consideration. 

The proofs of Theorem 16.41 and Corollary 16.71 are omit- 
ted, while the proofs of the other results are given in Sec- 
tion ivnEHl 

C. A General Scheme for Constructing Good LSCCs 

Theorems 16.21 and 16. 1 01 do give possible ways for construct- 
ing asymptotically good LSCCs. However, such constructions 
are somewhat difficult to implement in practice, because 
the generator matrices of a Gabidulin code or F^^ are 
not sparse. Thus, our next question is how to construct 5- 
asymptotically good LSCCs based on sparse matrices so that 
known iterative decoding procedures have low complexity. For 
such purposes, in this subsection, we shall present a general 
scheme for constructing ^-asymptotically good LSCCs. 

Let F = {Fk} ( ^L 1 be a sequence of random linear codes 
F k : X nk -> y mk . We say that F is (5-asymptotically SCC- 
good relative to the sequence {Ak}'^L 1 of subsets Ak C 

P„ fc (*)\{iV*}if 

limsup max — In a(Fk)(P, Q) < S. 
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Clearly, this is a generalization of (^-asymptotically good 
LSCCs, and may be regarded as an approximate version of 
(^-asymptotically good LSCCs especially when A% is a proper 
subset of V nk (X)\{P^ k }. The next theorem shows that 5- 
asymptotically good LSCCs may be constructed based on 
these linear codes by serial concatenations. 

Theorem 6.11: Let {G k } k K ^ 1 be a sequence of random 
linear codes G k ■ y mk — > Z lk that is <5-asymptotically 
SCC-good relative to the sequence {A k } k K L 1 of sets A k C 
Pm k (y)\{flr»*}- If there is a sequence F = {F k }™ =1 of 
random linear codes F k : X nk — > y mk such that 

F k (x n «\{o n «}) c (J r; ik (y) m 

PeA k 

then we have 

limsupp(G fc o S mfc o F k ) < . 
fc->oo UK* ) 

Remark 6.12: Take, for example, 

A k = {Pe V mh (y)\P(0) e [0, 1 - 7]} (68) 

for some 7 G (0,1)- Then Theorem 16.111 shows that we 
can construct asymptotically good LSCCs by a serial con- 
catenation scheme, where the inner code is approximately S- 
asymptotically SCC-good and the outer code has good distance 
properties. As we know (see 0, 0, 03. EH, etc.), there 
exist good LDPC codes over finite fields such that d67| i is met 
for an appropriate 7, so the problem to be solved is to find a 
sequence of linear codes that is <5-asymptotically good relative 
to a sequence of sets such as d68l ). In the next section, we shall 
find such candidates in a family of codes called LDGM codes. 

More interestingly, since an injective linear code F k always 
satisfies the condition d67l i with A k = V 1lk (X)\{P^ k }, we 
immediately obtain a corollary from Theorem 16.1 II 

Corollary 6.13: Let {G k } k x L 1 be a sequence of 5- 
asymptotically good random LSCCs G k ■ y m ' k — > Z k . Let 
F = {F k }^ =1 be a sequence of injective random linear codes 
F k : X n « -> y m ". Then 

lim sup p(G k o S mfc o F k ) < — — . 

fc^oo m.* ) 

Analogously, we have the following proposition. 
Proposition 6.14: Let {-Ffcj-fcLi be a sequence of 5- 
asymptotically good random LSCCs F k : X" k -> y m *. Let 



{Gk}kLi h e a sequence of surjective random linear codes 
G k : y m " -> Z l ". Then 

limsupp(G fc o S mfc o F k ) < 5. 

k— > 00 

The above two results tell us that, any linear code, if serially 
concatenated with an outer injective linear code or an inner 
surjective linear code, will not have worse performance in 
terms of the condition d52l i. Maybe, if we are lucky, some 
linear codes with better joint spectra can be constructed in this 
way. Note that a nonsingularrate-1 linear code is both injective 
and surjective, so adding rate-1 linear codes into a serial 
concatenation scheme is always a not-bad idea for constructing 
good LSCCs. Certainly, the addition of rate-1 codes may have 
a negative impact on the decoding performance. 

The proof of Corollary 16.131 is omitted, while the proofs of 
the other results are given in Section IVIII-II 

VII. An Explicit Construction Based on Sparse 
Matrices 

In light of Remark 16.121 we proceed to investigate the joint 
spectra of regular low-density genrator matrix (LDGM) codes 
over finite fields. In this section (except Proposition 17.81 ). we 
only deal with codes over a finite field ¥ q , so for convenience, 
we omit all subscripts of spectra or spectrum generating 
functions. 

At first, let us define three basic linear codes. 

A single symbol repetition code with a parameter c G N is 
a mapping /™ p : F 9 — > given by x i-> (x, . . . , x). 

A single symbol check code with a parameter d G N is a 
mapping f^ K : — > ¥ q given by x h-> ^ i=1 Xi, where 
the sum can be abbreviated as (x) ffi . In the sequel, we shall 
use this abbreviation to denote the sum of all components of 
a sequence (or vector). 

A single symbol random multiplier code is a random map- 
ping F^ M : F q — > W q given by x H> Cx where C is an 
independent random variable uniformly distributed over F£. 
Note that F^ M has the following simple properties: 

oP™ = if M Va G F g x 
1 q q 
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Based on the above codes, a random regular LDGM code 



F, 



LD 



q .c.a,n over F g is a mapping F^ u d/i 
by 

LD A f CHK „ 77R.M 



F' 



F= n denned 



*q,c,d,n — Jq,d,c'n P q,cd'n ^cd'n Jq,c,d'n \ m ) 

™h oro f REP £ rV l fREP fCHK ± rV l fCHK pRM A 
wnere J q ^ n — *0*i=l Jg,c > — \Ui=iJq,d > r q,n — 



(70) 



(c, g() ' (c, d) 

and (c, d) is the greatest common divisor of c and d. 

To calculate the joint spectrum of F^ d n , we first need to 
calculate the joint spectra of its constituent codes. We note 
that the definition d69l can be rewritten as 



LD 



ii pCHK y pw 
q,c,d,n ~ r q,d,c'n ^cd'n u r q,c,d'r 

d nCHK 



r REP 
q,c,d' 

pREP 



(71) 



r q,d,c'n ^cd'n Jq.c.d 

A f CHK n y n pREP 
— Jq,d,c'n ^cd'ri r q,c,d' 

where F REP = P RM o f REP o F RM F ™ £ F RM o f C ? K o 

vv ^ ^ <7,c,n q.cn J q,c,n q,n ' q,d,n q,n J q,d,n 

Fq^dn- Then it suffices to calculate the joint spectra of / REP 
and -Fg eTri . ^ e nave tne following results. 
Proposition 7.1: 

0(/ g T)Kv) = i]>>X 



aGF„ 



£[S(F REP )( U ,v)] 



-u w + -((u)e - u ) — 

q q V q - 1 



<?(/^n)(u,v) = J] ^(^)(^)U" P V» 

Pev n {¥ q ) 

S(f™*)(P,Q) = S(¥^)(P)l{P = Q} 
S(f^)(Q\P) = l{Q = P}. 
Proposition 7.2: 



cP 



(72) 

(73) 
(74) 

(75) 
(76) 



E[g(F™ K )(u,v)\ 



7 d+l 



((u)e) d (v)e + 



/ gup - (u) e 
V q-l 



(qv - (v) ffi ) 

(77) 



E[S(F^)(P, Q)] = cocf (gW „(u, Q), u d " p ) (78) 



VO e P d „(F 9 ) (0(a) > 0,Va 6 {o|P(o) > 0})(79) 
lna(F 9 c ™)(P,g) <5 gid (P(0),Q(0)) + dA (? , dn (P) (80) 



where 



((u)e) d + (,-D (<^_^)' 



iQ(0) 



((u) 



^ / quo - (u) e 



qn(d+l) Qdnl 



V 5- 1 



1 + (<?-!) 



d1 n(l-Q(0)) 



(81) 



f gO(O) 



, f gQ(Q)-i 

V 9-1 



V 3-1 

d1 n(l-Q(0)) 



5 q ,d{x,y)= inf 5 gd (x,x,y) 

0<a:<l 

5 g ,d(x, x, y) = dD(x\\x) + J q , d (x, y) 



(82) 

(83) 
(84) 

(85) 
(86) 

By Propositions 17.11 and 17.21 we obtain a tight upper bound 



J q A x ->y) = 



1 + («-!) 



+ (1 - y) In 



ga; — 1 
q-l 
f qx-1 

U-i 



A,,„(P)^ff(P)-iln( " 
n VnP 



on the joint-spectrum performance of F^ d , 



Theorem 7.3: 



E[S(F^ d>n )(Q\P)} = E[S(F™* n )(Q\P)} (87) 



— lna(P^J(PQ) < -^, d (P(0),Q(0)) + cA g , c , dn (P) 

(88) 

where c' and <i' are defined by ( l70t . 5 g> d(x,y) is defined by 
(13, and A g , n (P) is defined by (|86>. 

To have further insights into this bound, we analyze the 
properties of S qtd (x,y). 

Lemma 7.4: For all x,y E [0, 1], 



s q ,d( x >y) < JqA x ^y)- 

Lemma 7.5: For all x,y E [0, 1], 



■/ 9 ,<f (a, y) < In 



1 + («/ 



(89) 



(90) 



Based on the results above, the next theorem shows that 
for any S > 0, regular LDGM codes with d large enough are 
approximately <5-asymptotically SCC-good. 
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Theorem 7.6: Let {F^ ° d n }^—i be a sequence of random 
regular LDGM codes with the constant coding rate r$ = d/c. 
Let {A n }^ =1 be a sequence of sets A n C Vd'n@? q )\{P d> n }. 
Then 

limsup max J- In a(F^ dn )(P, Q) < Po (91) 

n— >oo P£LA n . (I ft 

where c' and d! are defined by (170t . and 



po = lim sup max — In 



1 + («-!) 



gP(0) - 1 



If 



A n = P e V d>n {V q ) 



P(0)G 



1 1 

7i> - 

q q 



12 



where 7i e (0, l/g]\{|} and 72 G (0, (g - l)/q), then 



po = — In 

ro 



1 + (?-!) 



g"7 



(92) 



(93) 



(94) 



(95) 



where 7 = max{7i,72}. For any S > 0, define 

Then we have p < ^ f° r a U ^ > ^«(7i 

Theorem 17 .6 1 to gether with Theorem 16.111 and Remark [6. 121 
shows that, for any S > 0, we can construct (^-asymptotically 
good LSCCs by a serial concatenation of an inner LDGM 
code and an outer linear code with good distance properties. 
In particular, we may use LDPC codes as outer codes. Fur- 
thermore, Proposition 15.71 shows that we can find a sequence 
of asymptotically good LSCCs in a family of sequences of Si- 
asymptotically good LSCCs, where <5, is decreasing in i and 
converges to zero as i — > 00. An analogous construction was 
proposed by Hsu in his Ph.D. dissertation |[T8l . but his purpose 
was only to find good channel codes and only a rate-1 LDGM 
code was employed as an inner code in his construction. 

Next, we give an example to show how we determine the 
parameters of the inner LDGM code when designing such 
codes. 

Example 7.7: Let 5 = 0.05 and f n : F£ -> F|" an injective 
linear code. Suppose that the normalized weight of all nonzero 
codewords of /„ ranges from 0.05 to 0.95. We shall design 
a linear code H„ : -> ¥\ n that is 6-asymptotically SCC- 
good. Let H n = G n o Es„ o /„ where G n is a random regular 
LDGM code over F2. It is clear that the coding rate of G„ 



must be |. Using Theorem 17.61 with q = 2, r = |, and 
7i = 72 = 0.45, we have d q (^, 5R(f n )) = 35. Then we may 
choose LDGM codes with c = 14 and d = 35, so that po < 
0.01, and therefore lim sup n _ J . 0O p{H n ) < Po/R{fn) = 0.05. 

So far, we have presented two families of good linear 
codes, one based on Gabidulin codes (Section |VI-A| i and the 
other based on LDGM codes. It is necessary to give some 
comparisons of these two families of codes. At first glance, it 
seems that the family based on Gabidulin codes is better than 
that based on LDGM codes since the former is SCC-good 
while the latter is only asymptotically SCC-good. However, 
this is not the truth, because there is no single linear code 
that is SCC-good. The proposition that follows states the fact. 
Consequently, in terms of code spectrum, the two families have 
almost the same performance. On the other hand, in terms of 
decoding complexities, the family based on LDGM codes is 
more competitive than that based on Gabidulin codes, since 
LDGM codes as well as LDPC codes are all characterized 
by sparse matrices so that low-complexity iterative decoding 
algorithms can be employed. 

Proposition 7.8: For any linear code / : X n — > y m with 
\X\ > 2, 



} a(/)(P,Q)>e(m^ i ) 
The proofs are given in Section IVIII-JI 



max 

per n (x)\{p ^ 
Qev m (y) 



VIII. Proofs 



A. Proofs of Results in Section \IV-A\ 



Proof of Proposition \4.2\ It is clear that, for any xj ?r 
and xj m satisfying P Xi = P%. for each i e I m , 



E 



l{x Im e A} 
\A\ 



= E 
{ ^E 
= E 



l{x Im 6 A} 
\A\ 

l{xi m 6 A} 
\A\ 

l{x Im 6 A} 
\A\ 
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where (a) follows from the property that the distribution of A 
is invariant under any permutation. Then it follows that 



E 



l{x Im e A} 



\A\ 



n£ii7£(*)i 



E 



E 



nr=ii7?'.(^)i 



E 



E 



1{xi,„ 6 A} 
\A\ 

l{± Im € A} 



1 



Ili=i ( ni P x ) 

1 



E 



E 



IL=i (m 

(a) a(A)((P x Ji 6X , 



14 



nti\^\ nt 

where (a) follows from the definition of joint spectrum and 
Proposition 13.11 This concludes ( fTTT ). and the identity ( fT~4T > 
comes from 



E 



E 



l{(xi,,x l7iAl! ) e 1} 



E 



14 

UCxi,,^^,) G A} 



combined with (fTTT i and Proposition 13.11 



□ 



B. Proofs of Results in Section [IV-B\ 
Proof of Proposition \4.4\ 



3=1 



E 



E 



3=1 



E 



l{(xi ; ,yi m )erl(F)} 



yx m en J fc =1 r Q /(y J ) 



E 



E 



E lUx^yxJerKF)} 



)] 



= E[S yi ...y klXl ... Xl (F)(Q Ik \(P Xi ) ieXl )] 



where (a) follows from Proposition 14.21 and the identity 



iri(ioi = nui*ii n< - 

Proof of Proposition 



□ 



E[S z{x {Goi: m oF){Q\0)} 

^{(^or)(x)er^)} 

Pe~P m (y) 



5] Pr(x)e7?(^)} 
x P{G~(P~(x)) G TA(Z)|P~(x) g r P m (y)} 



( = } X! S[ 1 S 3 ;|^(P)(P|0)] J B[5 2 | 3 ;(G)(Q|P)] 

where (a) and (b) follow from Proposition 14.41 and x is an 
arbitrary sequence such that P x = O. □ 



C. Proofs of Results in Section \IV-C\ 

Proof of Proposition \4.7\ At first, we explicitly write 

K = OlLi F(l) and G ra = QT= i G W where and G W 
are the ith independent copies of F and G, respectively. 

Next, let B' = (F n G m )(B). Since both F n and G m are 
bijective, the generating function Q(B')(u, v) can be rewritten 
as 



£W(u,v) 

i^l{(x,y)eB'K F]I v^ 



^ 7i m 

= TbT E y) e p} J] u FW(Xi) n « G0 ) 

x,y i=l 3=1 

Taking the expectation on both sides, we obtain 



E[g(B')(u,v) 



= E^ 



i{(x,y) eP} 
IPI 



3=1 



E[g(B)((E[u F{a) ]) aeX ,(E[v G{b) }) bey )] 
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which is just the expectation E[Q(B)(u, v)] with substitutions 
(O and (EOj. □ 



D. Proofs of Results in Section [IV-D\ 

Proof of Proposition \4.8\ By the definition of spectrum 
generating functions, we have 

1 



5(4xB)(u,v) = 



\A x B\ 



u nf,3c v mPy 



(x.y)eAxB 



\A\ ^ B ^ 

1 1 xeA 1 1 yes 

= g(A)(u).g(B)(v). 



□ 



Proof of Proposition \4. 9\ Since U 2 is a refinement of 1A\, 

g X u 2 (A)(n 2 ,u 2 ) = p| n u sr x2 



4e n n 4' 



Ml ^ XI ll ^ 

C/ 2 CC/i 



r.- 2 



Applying the substitution ( 1231 ) and the identity 

J2 \U 2 \P^(a) = \U 1 \P^(a) Vae* 
(7 2 ew 2 , 

U 2 CU! 

we obtain the generating function Q x u x (A)(ui,u 1 ). 
Proof of Corollary WM 

Qx{Ai x A 2 )(u) = x A 2 )(u,u) 

^e*(Ai)(u)-0*(.l a )(u) 



□ 



where (a) follows from Proposition 14.91 and (b) follows from 
Proposition [4J] □ 
Proof of Corollary \4.12\ 

(a) 

foy(/i®/2)(u,v) = Qxyxyifi © /2)(u, v,u,v) 

= ^y(/i)(u,v) • foy(/ 2 )(u,v) 

where (a) follows from rl(/i 0/ 2 ) C X ni x X 712 x y mi x 
and Proposition 14.91 with the default partition, and (b) follows 
from Proposition with rl(/i / 2 ) = rl(/i) X rl(/ 2 ). □ 



E. Proofs of Results in Section [IV-E\ 

To prove Theorem 14.131 we first need two lemmas. 
Lemma 8.1: Let A be a subspace of F™. Then 

^- • x 2 ) = l{x 2 e A^} Vx 2 e FJ. (96) 



Proof: For a fixed x 2 , the mapping r : A — > ¥ q given by 
x h-> x-x 2 is an F 9 -module homomorphism of A into F 9 , and 
hence the image set t(A) C F g is also a vector space over 
F g , which must be either or ¥ q . 

If x 2 e A- 1 , then x(*i • x 2 ) = x(0) = 1 for all Xi e A, 
and hence the identity d96l > holds. If however x 2 g" A- 1 , then 



X! *( xi • X2 ) = \ E ^ = °- 

1 1 xiSA q ae¥„ 



The last equality follows from the property (1281 1. □ 
Lemma 8.2: Let U be a fixed partition of I„. Then 

for all xi G F™, where M is defined by 
Proof: 

2^ X(xi -x 2 ) _[]_ 



x 2 £F" 



ueu 



e n n x^m^^w,^,, 

x 2 gfj ueu ieu 

]][ [] E x(xi^a 2 )u Uy a 2 



ueu ieUa 2 ew q 

II II E x(aia 2 )u Uy a 2 
ueu ai ev„ \a 2 eF„ 



l^l-Pxi(oi) 



ueu 

where (a) follows from the identities 

n n 

X(xi -x 2 ) = xC^2 x is x 2s) = X\x{xisX2s) 



and 



n uu ' : 



ieu 



Proof of Theorem \4.13\ 

g ¥ u{A^){xi u ) 

1 \- TT MP* 

AM E [l u u 



\A±\ 



xjfEA 1 ueu 



jjt\ E H^eA^Hu^ - 



x 2 GF? 



ueu 



(a) 1 



E TTT E 



|A||A^| 



Xi -X 2 j J| 

x : J ;; 1 " x , ... 1 ueu 

u 



IA-H ^ \A\ 
1 1 X2 eFj 1 1 

OXi E E X(xi-x 2 ) J] u, 



□ 



xieAx 2 eF» 
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\A\\A±\ 



1 



■g x u(A){{uuM) UeU ) 



where (a) follows from Lemma |8T1 and (b) from Lemma 18721 

□ 

Proof of Theorem \4.15\ Define the sets 

Z 1 ^{(xA,x)eF" l xF"|xGF"} 



and 



Z 2 = {(y, -yA T ) G F™ x ¥ n q \y G F™}. 



Clearly, for any z x = (xA,x) G Z\ and z 2 = (y, — yA T ) G 
Z 2 , we have 

zi • z 2 = (xA) • y + x • (-yA T ) 
= (xA)y T - x(yA T f 
= xAy T — xAy T 

= 

which implies Z 2 C Z^. Note that |Zi||Z 2 | = q m+n . This, 
together with the identity \Zi\\Z±\ = q m+, \ gives Z 2 = Z±. 
Then it follows from Theorem 14.131 that 

g e v ¥ u(-g)(-v v ,u u ) 

= g r v r u(Z 2 )(v V ,Uu) 
= Gwv ¥ u(Zi )(v V , u u ) 



1 



.W(Zl)((vyM)y eV , (UuM) UeU ) 



\zt 

—g r u r v(f)((u u M) U€U , (v v M) VeV ) 



□ 



as desired. 

Sketch of Proof of Theorem \4.16\ Establish an analogue 
of Lemma [8TI by using the fact that the kernel of a generating 
character contains no nonzero ideal. Then use an argument 
similar to that of Theorem 14.131 □ 

F. Proofs of Results in Section [F| 



Proof of Proposition 15.71 For any e > 0, define the 
sequence of events 



Ai 



\oj G fl 



\RJF? 



Rs(F)\< £ -} 



It is clear that Mmk^-oo P(A k ) = 1, so that the conditions d40l ) 
and d4~n > hold. Furthermore, we have 
1 



max — In P{x G ker F k \ Ak } 

x:x#0™* n k 

= max — In E 
xix^o™* n k 



X\ nk l{x G keri^~} 



F fc ~(^)||ker^ 



-4, 



< max — In 

x:x^0"f= n k 

< -R s (F) + l 



+ max — In 

x:x^0"fc n k 

(a) / s 2e 
< -R S (F) + - 4 



\X\ nk 



e n k (R.(F)-% 



-E 



l{x G ker F k } 



IkerF* 



A k 



\X\ 



P(A k ) 



E 



l{xGkcrF fc } 
I ker F k I 



max — In a (ker F k ) (P) 

P£-P n Ax)\{P n k } n k 



(b) 
< 



-Rs(F) 



for sufficiently large k, where (a) follows from Proposition |4.2| 
and \im k ^ oo P(A k ) = 1, and (b) follows from (1501 1. This 
concludes d43l and hence proves the proposition. □ 

Proof of Proposition \5.2\ Use argument similar to that 
of Proposition 15.11 □ 

Proof of Proposition 15.31 Apply Proposition 13.21 □ 

Sketch of Proof of Proposition \5.4\ Using the Rronecker 
decomposition theorem, we can always find a group X lk for 
some l k and a random linear code G k : X lk — > X nk (does not 
have to be injective) such that G k (X ln ) = ker F k , and hence 
G k is a (5-asymptotically good LCC. □ 

Sketch of Proof of Proposition \5.5\ Using the Rronecker 
decomposition theorem, we can always find a group y lk for 
some l k and a random linear code G k : y nik — > y k such that 
kcrGfe = F k (X nk ), and hence G k is a (5-asymptotically good 
LSC. □ 

Proof of Proposition \5.6\ It follows from d52l ) that 

max — ]na(F k )(P,Q) < 5 + e (97) 
Pe-P nk (X)\{p„ nk } n k 

QeTm k (y) 

for any e > and sufficiently large k. Then it follows that 

max — In afker F k )(P) 

Pev„ k (x)\{P ank } n k 



(a) 



max — In E 

x:x^0"t n k \ 

1 



|A?| n *l{x G kcrffe} 
|kerF fe | 



< max _ In(|yr*P{K w (x) =0 mfc }) 

x:x^0"fc 7l k 

= max — ln(\y\ mk P{F k (x) = m *} 

x:x^0"fc n k V 
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(b) 



max — ]na(F k ){P, P ™ k ) 
PeV n .(x)\{P n k } n k 



< 6 + e 



for sufficiently large k, where (a) follows from Proposition l4.2l 
(b) from Proposition 13.21 and (c) follows from d97l ). Since e 
is arbitrary, we conclude that F is <5-asymptotically SC-good. 
Also by Wl\ , we have 

E[S X y(F k )(P,Q)] < e n *^S xy (X n * x y m *)(P,Q) 

vp g v nk (x)\{p ^},yQ g v mk (y) (98) 

for sufficiently large k. Then for any Q G P mk (y)\{Po™k }, 
it follows that 

E[S y (F k (X n *))(Q)} 

( => E[S y (F k )(Q)} 

= Y, E[S xy (F k )(P,Q)] 
Pev n . (X) 



O) 



Y E[S xy (F k )(P,Q)} 
Pev nk (x)\{p n k } 



(c) 

< 



Y e nk{S+e) S xy {X nk x y m "){P,Q) 

Pev nk (x)\{p„ nk } 

< e nk{S+c) Sy{y mk )(Q) 

for sufficiently large k, where (a) follows from the property 
of linear codes, (b) from the condition Q ^ Pvi m k, and (c) 
follows from ( |98l . Hence we have 

limsup max — hxa(F k (X nk ))(Q) 
fc^oc Qev mk (.y)\{p m k } m k 

1 E[S y (F k (X m *))(Q)] 
= lim sup max In ; — — 

fe^oo Qev mk (y)\{P omk } m k Sy(y m ")(Q) 

< limsup — In e™ fc(,5+e) 
fe^oo m k 

= (S + e)R(F). 

Because e is arbitrary, F is <5i?(i r ')-asymptotically CC-good. 

□ 

Proof of Proposition 15. 71 Note that 

limsupp(G fcife ) < limsupp(G life ) < Si, Vi G N 

fe— 5-00 fc— J-OO 

and hence limsup^^^ p(G k ,k) < inUen Si = S. □ 

G. Proofs of Results in Section IV7-AI 

Proof of Proposition \6.1\ We may assume n > rn and 
C = Cfc for some 1 < k < m. Since x\, . . . ,x m can be 



X m 

' 8 



C / g jpnxr. 



extended to a basis iei, . . . , x n of ¥ qn over F g , the code C k C 
is obtained by projection from a corresponding code 
. Since the corresponding properties of C' k clearly 
imply those of C k , it suffices to consider the "square case" 
in = ti. Since C\ C C2 C • • • C C n , we may further restrict 
attention to the (n,n,n) MRD code C\. 

In this case, since \C\\ = q n , it suffices to show that the 
maps Ci — > F™, A H» xA and A n- Ay T are one-to-one. 
Now xA = xB (A, B G d) implies x(A - B) = 0" 
and hence rank(A — B) < n. Since C\ has minimum rank 
distance n, this implies A = B, proving the claim. The case 
of A H» Ay T is done in the same way. □ 
Proof of Theorem \6.2\ We shall show P{F(x) = y} = 
q~ m for all x e F™ \ {0"} and y G F™; equivalently, for fixed 
x G F™ \ {0™} the number P{F(x) — y} is independent of 
y. By definition of F, 

= y) = — £ ~j^p = ~ - 

By Proposition 16.11 the linear map C — > F™, A M> xA is 
surjective. This implies that |{A G G|xA = y}| is equal to 
the cardinality of the kernel {A G G|xA = 0™ 1 } and hence 
independent of y. □ 

H. Proofs of Results in Section IW-BI 

Proof of Theorem \6.5\ Condition (l55l l implies that for 
every pair x G X n \ {0 n }, y G y m there exists at least 
one linear code / : X n — > y rn satisfying /(x) = Since 
m, n > 1, this in turn implies, for every pair x G X \ {0}, 
y G y, the existence of at least one group homomorphism 
h : X — >• y satisfying h(x) = y. (In fact the abelian group 
Hom(A'", y m ) of all group homomorphisms from X n to y m 
is isomorphic to the group of all m x n matrices over the 
abelian group H := Hom(X,y) and the order of H can be 
computed from the structures of X and y.) 

If p is a prime dividing \X\, there exists x G X of order p. 
Then, by the condition above, every y G y must have order 1 
or p, so y = for some s. If A" has a prime divisor q^p, 
then y == = and so |^| = 1, a contradiction. Thus, <Y 

2 The existence of such an / depends only on the types of x and y, since 
/ is linear iff a o f o ix is lineal' for any permutations ?r 6 S„, o" £ S m . 
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and y must be p-groups for the same prime p, and y must be 
elementary abelian. Finally, if X contains an element of order 
p 2 and h : X — > y is any group homomorphism, we have 
px ^ but h(px) = ph{x) = 0. This implies again \y\ = 1 
and concludes the proof. □ 

To prove Theorem 16.61 we need the following lemma. 

Lemma 8.3: Let F : X n — > y rn be a random linear code. 
If P is SCC-good, then 



£[|kerF|] = 1 + \y\ 

Proof: 

E\\kerF\] = MlkerFll 



\X\ n -l) 



(99) 



= E 
= 1 

(a) 



E 1{F(X) = 0™} 

E P{H*) = 0™} 

x6A-"\{0"} 



i + \y\- m (\x\ n - 1) 



□ 



where (a) follows from (|55] > 

Proof of Theorem \6.6\ By Lagrange's theorem, |kerF| 
can take only values in {l,p, . . . ,p rn }. Hence, using 
Lemma [iOl we obtain 

q n - 1 



1 



= P[|kerF|] 
>P{|kerP| = 1} +p- (1 - P{|kerP| = 1}) . 



Solving for P{|kerP = 1} gives the stated inequality. □ 
Proof of Proposition \6.8\ Suppose X = Z|, so that 
X n = Z% s = (F£ s , +) for all n G N. Let F n : ¥% s -> F^ s 
be the random linear code derived from a binary (ns, ns, 2) 
Gabidulin MRD code C in accordance with Theorem 16.21 By 
definition, the code C consists of 2 2ns matrices A € jr™ 3 *™' 5 
with rank(A) e {ns — 1, ns} and hence |kerA| G {1,2}. For 
P„ this means that equality holds in the bound of Theorem l6.6l 
and we get 

2-2 + 2~™ s 



lim P{|kerP„ = 1} 



lim 

n— »oo 



0. 



2-1 

Since P n is SCC-good (see Theorem 16.21 ), this proves the 
proposition. □ 
Proof of Theorem \6. 9\ The inequalities ( f62b and d64j > are 
easy consequences of Theorem 16.41 Next, let us evaluate the 
average conditional spectra of G\ and G 2 - 



For any P G V n (X)\{P on } and Q G P m Q>), 

B[53;^(G 1 )(Q|P)] 

= E E[S ylx (F)(0\P)}E[Sy ly (F^ C m )(Q\0)} 
o<ET m (y) 

= HQ ^ P ™}E[Sy(y n )(Q)} 

E f[^i^(^)(oip)] 

oe-p m (y)\p om 
+ 1{Q = Po™}^[^^(P)(Po™ |P)] 

+ £[s y (y")(Q)] E s[^|^(P)(0|P)]j 

oe-p,„(y)\p om / 

= £[Sy(y i )(Q)] E F[^|^(P)(0|P)] 
oe-p m (y)\p ™ 
+ 1{Q = P om }E[S ylx (F)(P om \P)] 

< E[S y (y n )(Q)\ + 1{Q = P 0m }E{S ylx (F)(P 0m \P)} 

where (a) follows from Proposition 14.61 and (b) follows from 
d54b . This concludes (l63l . 

Analogously, for any P G V n (X)\{P n} and Q G V m (y), 

E[S ylx (G 2 )(Q\P)] 

® E S[^|^(^)(0|P)]P[^|^(P)(Q|0)] 
oev„(x) 

( => £ E[5*(* n )(0)]S[S y ,^(F)(Q|0)] 
oePnW 

= E £[S*y(f)(O,0)] 

- £?[5y(F)(Q)] 

= P[^(P(A-"))(Q)] 

where (a) follows from Proposition 14.61 and (b) follows from 
d54| l. This concludes (|65l l and hence completes the proof. □ 
Proof of Theorem \6.10\ For the first statement, recall that 
the source transmission rate of (^-asymptotically good LSCs 
must converges. Then for any e > 0, since / is asymptotically 
SC-good and satisfies d66l >. we have 

a(ker/ fe )(P) < e n ^ &+ ^ VP G P rik (X)\{P »> } (100) 

and 

\fk{X n ")\ > \y\ mk e~" lke > \y\ m * e - 2n ^/ R (f) (101) 

for sufficiently large k. Because y is elementary abelian, we 
define the random linear code G\.k = P^m^ mfc ° fk, where 
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q = \y\. It follows from Theorem 16.91 that 



inf P{keiG hk = ker/fe} > K q 

fcGN 



(102) 



and 



E[S y \ x (Gi, k )(Q\P)] 

< Sy(y m *)(Q) + 1{Q = P m h }Sy lx (f k )(P m k \P) 

vp g v nk (x)\{p ^},VQ e v mk {y). 

Hence for any P G T Uk (X)\{P n k } and any Q G T mk (y), 

a(G hk )(P,Q) 

^ E[Sy lx (G hk )(Q\P)} 

s y (y m *)(Q) 

< l + l{Q = P ™ k }a(f k )(P,P omk ) 

( => i + hq = p omfe }|yr^-p{/r(x) = o™n 

= 1 + 1{Q = P omt }|yp»P{x g Wfe} 



( = ) l + l{Q = P omt }|^| 



|ker/ fc | 
|;tK 



a(ker/ fc )(P) 



I -Vim* 

= 1 + 1{Q = P om 4-^L_^ a (ker A)(P) 

< 1 + 1{Q = P ™ fc }e 2 " fce/fl(/) a(ker/ fe )(P) 

< e n fc (<5+2 C +2c/fl.(/)) 

for sufficiently large fc, where (a) follows from Proposition [372] 
and x is some sequence of type P, (b) from Proposition |4.2| (c) 
from ( 1101b , and (d) follows from ( 1 100b . Thus for sufficiently 
large fc, 



P (G 



l,k 



< 5 + 2e + 



2e 



Define the new random linear code G[ k as G\^ conditioned 
on the event A k = {kerGi^. = ker fk} (i.e., an expurgated 
ensemble). Then it follows that for sufficiently large k, 

p(G' hk ) < p(G 1>k )-— In P(A k ) 
n k 

(a) 

< 5 + 3e • 



2e 



oo 
k=\ 



where (a) follows from ( 1102b . Since e is arbitrary, [G' x k } 
is a sequence of (5-asymptotically good LSCCs such that 
kerG" lfc = ker f k . By Proposition 4.1 in |37|, we conclude 
that there exists a sequence {<?i,fc}fcjLi of <5-asymptotically 
good LSCCs g hk : X" k -> y m « such that g x>k is SC- 
equivalent to f k for each fcGN. 



The proof of the second statement is analogous. Let e > 
be given. Since / is ^-asymptotically CC-good, we have 

a(f k (X n *))(Q) < VQ G V mk (y)\{P Q ^} 

(103) 

for sufficiently large fc. Noting that X is elementary abelian, 
we define the random linear code G 2ik = fk° E^^ nk , wnere 
q = \X\. Then it follows from Theorem 16.91 that 



and 



inf P{G 2 , k (X n ») = fk(X^)} > K q (104) 

fceN 



E[Sy\ X (fr, k )(Q\P)] = Sy(fk(X n *))(Q) 

vp g r nk (x)\{p ~ k },vQ g v mk {y). 

Hence for any P G V nk (X)\{P n k } and any Q G V mk (y), 

(r upn , E[S ylx (G2,k)(Q\P)} 
a(G 2>k )(P,Q) = Sy{ymk){Q) 

_ Sy{f k {X n »))(Q) 

s y (y m »)(Q) 

= a(fk(X n »))(Q) 

for sufficiently large fc. Define the new random linear code 
G' 2 k as G 2 ,k conditioned on the event Bk = {G2,k(X nk ) = 
fk(X n ")}. Then it follows that for any P G V nh (X)\{P n k } 
and any Q eV mk (y)\{P ™*}, 



for sufficiently large fc, where (a) follows from J 103b and 
d 1 04b - Since fk is injective, we have 

«(^, fc )(P,Po^) = VP G V nk (X)\{P , k } 



and 



R{fk) 



Rc(fk) 
hx\X\ 



converges as fc — > oo. Therefore, we have 
p(G' 2}k ) <^jy+e 

for sufficiently large fc. Since e is arbitrary, {G' 2 kik^i i s a 
sequence of <5/P(/)-asymptotically good LSCCs such that 
G 2,k( Xnk ) = fk(X nk ). By Proposition 4.1 in [37], we 
conclude that there exists a sequence {g 2 ,k} k Li of 6/R(f)- 
asymptotically good LSCCs 172.*: : X nk — > y nik such that g 2t k 
is CC-equivalent to fk for each fcGN. □ 
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/. Proofs of Results in Section WI-C\ 

Proof of Theorem \6.11\ Let e > be given. Since G k is 
(5-asymptotically SCC-good relative to A k , we have 

E[S z \y{G k )(Q\P)\ < e m *V+^S z (Z l »)(Q) 

VP e A k ,Q e Pi k (Z). (105) 

for sufficiently large k. Then for all O G V nk (X)\{P^ k } and 

S z \ x {G k aY, mh oF k )(Q\0) 

® E E[Sy lx (F k )(P\0)]E[S zly (G k )(Q\P)] 

= E E[S y]x {F k )(P\0)]E[S z]y {G k ){Q\P)] 



PeA k 



(c) 



< £ e m ^S z (Z^)(Q)E[S y]x (F k )(P\0)} 

< e mkiS+ ^S z (Z lk )(Q) 

for sufficiently large k, where (a) follows from Proposition l4.6l 
(b) from the condition d671 i, and (c) follows from (|105l l. 
Therefore, for sufficiently large k, 

p{G k o S mfc o F k ) < -^il + e . 

Since e is arbitrary, this establishes the theorem. □ 
Proof of Proposition \6.14\ For any e > 0, since F k is 
(5-asymptotically SCC-good, we have 

E[S y]x {F k )(P\0)] < e n ^ s+ ^Sy(y m *)(P) 

VO e P„ k (Af)\{P » fc }, P e P mk (y). (106) 

for sufficiently large k. Then for all O £ V nk (X)\{P ™ k } and 

Q&Piuiz), 

S zlx (G k oX mh oF k )(Q\0) 



(a) 



E s [^l^(^)( p l )]^[^(G fe )(Q|P)] 
Pe7> mfe (30 



PePmj (30 

= e M*+e) £ P[<?^(G fc )(PQ)] 



PeP mfc (30 
S z (G k )(Q) 



= e n ^ s+ ^S z (G k (y m "))(Q) 
= e nkis+ ^S z (Z h )(Q) 



for sufficiently large k, where (a) follows from Proposition |4.6l 
(b) from ( 1106b . and (c) follows from the surjective property 
of G k . Therefore, for sufficiently large k, 

p(G k o £ mfc o F k ) < S + e. 

This concludes the proof, because e is arbitrary. □ 

/. Proofs of Results in Section WIH 

Proof of Proposition 17. 71 The identity d72l holds clearly. 
This together with Propositions 14.71 gives the identity ( |73T >. 
From (l72l and Corollary 14.121 it further follows that 



\ q ae¥ q J 

= - E 



VnP 



u nP v ncP 



PeP„(F,) 

= E <S(F")(P)u™ p v ncP 

This proves (174) . The identities (l75"l l and (F76b are easy conse- 
quences of (1741 . □ 
Proof of Proposition \7.2\ Note that the generator matrix 
of Fq ™ is the transpose of the generator matrix of F^f p . 
Then by Theorem 14.151 it follows that 

E[g(-F^ K )(u^)} 



= - d E{G(F«n(v,u)} 



(a) 1 



d+1 



q 

(b) l 



q 



d+l 



UqVQ 



(u) e - Up 
9-1 



((v)© - vq) 



((u) ffi ) d (v) ffi + 



qu - (u) 



(qv - (v)®) 



where u = uM and v = vM, (a) follows from Proposi- 
tion |TT] and (b) follows from the property ( 129) . This together 
with Proposition 14.71 concludes the identity ( 1771 ). 
By Corollary 14.121 we further have 

E[g(F^)(u^)} 

1 



i.(d+l) 



((u)er(v)e 



/ qu - (u) e 



V 9-1 
1 



(qv - (v)©) 



((u)ef + (?-!) 



gMQ - (u) ( 

9-1 



*'() 
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((u)e)' 

d+1) 



d (Quo - (u)p 



n 



((v)e - v ) 



q n(d+l) ^ | I n Q 



((u) e ) d + (g-l) 



/ qu a - (u) 



V 9-1 



((u)e)° 



d / gu - (u) { 



n(l-Q(0)) 



Hence, 



S[5(F g c ™)(P,Q)] = co C f(S[g(F g c ™)(u,v)],u d " p v"Q) 
= coef( 5 W n (u,Q),u d ^) 

where g^ 1 ] n (u, Q) is denned by ((ST). This proves d78l l. 

Since 9 q l \ n {^Q) is a polynomial with nonnegative coef- 
ficients, coef (g^lt „(u, Q), u dnP ^j can be bounded above by 



coef(^L(u,Q),u d " 



^ < &n(0,Q) 



QdnP 



Note that D{P\\0) > D(P(0)||O(0)) with equality if and 
only if P(a)(l - O(0)) = 0(a) (1 - P(0)) for all a ^ 0, and 
thus we obtain a minimized upper bound (f80b . □ 
Proof of Theorem 17.51 By the definition of F^ d n and 
Proposition 14.61 it follows that 

E[S(F™ d<n )(Q\P)} 

= J2 E {S(G n )(0\P)}E[S(F^X)(Q\ )] 



(a) 



E 

oeP cd -„(F,) 

CHK 



i{o = p}p[5(p^ h c \)(0|o)] 



= E{S(F^X)(Q\P)} 

where (a) follows from d76b . This concludes d87b . 
Furthermore, we have 



^WP^ in )(P,Q) 
= J_, E[S(F^ n ){Q\P)} 

d'n n S(Ff l )(Q) 
(a)J_ E[S(F^X)(Q\ P )} 
d'n n S(F§'«)(Q) 



1 



In a(F^X)(P,Q) 



where O is an arbitrary type in Pd„(F g ) such that 0(a) > 
for all a G {a|P(a) > 0}, and g q 2) dn (0,P,Q) is defined dy 
|l. This gives ( T79b . 
Finally, let us estimate a(P^™)(P, Q). 



< 



S(Ff xF«)(P,Q) 
1 



/ tin ' 
Un-P. 



QdnP 



+ (.-i)(^fi)' 



1 / gO(0)-l 
V 9-1 



d1 n(l-Q(0)) 



dnH(P) pdnP 



\dnP 



1 + («-!) 



/gO(Q) - 1 



f gO(Q) - 1 
V 9-1 



V 9-1 

»(1-Q(0)) 



nQ(0) 



e dnA, i(i „(P) e <inD(P||0) 



1 + 



iQ(0) 



1 / gO(Q)-i 



d1 n(l-Q(0)) 



< -[^, d (P(0),Q(0)) + dA g , c ^ n (P)] 
= ^ >e2 (P(0),Q(0)) + cA 9 , c , dn (P) 

where (a) follows from (T87T > and (b) follows from ( f80b . This 
concludes d88l l and hence completes the proof. □ 
Proof of Lemma \7J\ When x G (0,1) and ?/ G [0,1], 
the inequality d89l l clearly holds by taking i = x in d83l ). If 
however x = 0, then 

1 



lim <5 9i(i (0,i,y) 



lim In ■ 

x^0 \ 1 



J q ,d(x,y) 



Hence 5 q ,d{0,y) < J q .d(0,y). Similar arguments also apply 
to the case of x = 1. □ 
Proof of Lemma [73} Apply Jensen's inequality. □ 
Proof of Theorem \7.6\ By Theorem l7.3l Lemmas 17.41 and 
17.51 and the condition ro = d/c, it follows that 



1 



7-Wi^ d ,J(P,Q) 



< — In 

H) 

(a) 1 

< — In 

?"0 



1 + (qQ(0) - 1) 



( qP(0) - 1 
V 9-1 



1 + (9 - 1) 



qP(0) - 1 



9-1 



+ C Ag, c / rfn (P) 

§ln(cd'n + 1) 
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where (a) follows from the strict increasing property of In x, 
Q(0) e [0,1], and the inequality 

" > - — 1 —e nH{p) (see @ Lemma 2.3]). 



y nPJ ~ (n + 1)1 
Note that 

Mm qHcd 'r +1) = 0. 

All conclusions of the theorem follow immediately. 
Proof of Proposition \7.8\ 

max a(f)(P,Q) 
Qer m (y) 



□ 



> max 



Sy\ X (f)(Q\Pa») 



> 



Qev m {y) S y {y™){Q) 
max Qg-p m (y) Sy\x{f){Q\P a ™) 
maxQ e7 ,„ (y) Sy(y m )(Q) 

\y\ m 



(a) \y\ m 



B TO 2 



where a ^ and (a) follows from Stirling's approximation. 

□ 

IX. Conclusion 

In this paper, we present some general principles and 
schemes for constructing linear codes with good joint spectra. 

In Section [VI- Al we provide a family of SCC-good random 
linear codes which are derived from Gabibulin MRD codes. 
In Section IVI-B1 it is proved in Theorem 16.101 that, when 
the output (resp., input) alphabet is elementary abelian, we 
can construct (5-asymptotically good LSCCs which are SC- 
equivalent (resp., CC-equivalent) to given <5-asymptotically 
good LSCs (resp., LCCs). In Section IVI-C1 we further propose 
a general serial concatenation scheme for constructing good 
LSCCs by Theorem 16.1 II The interesting role of nonsingular 
rate-1 codes is also discussed in Corollary 16. 131 and Proposi- 
tion |6J~4] 

In Section IVHI joint spectra of regular LDGM codes are 
analyzed. By Theorem 17.61 we show that regular LDGM 
codes with appropriate parameters are approximately 5- 
asymptotically SCC-good. Based on this analysis, we finally 
present a serial concatenation scheme with one LDPC code 



as outer code and one LDGM code as inner code, which is 
proved to be asymptotically SCC-good. 

Besides the major contributions above, we also develop the 
code-spectrum approach and review in a uniform viewpoint 
the problems of constructing good linear codes for different 
applications. In Section [IV] we establish a series of new 
concepts and results of code spectra, among which, spectrum 
generating functions, spectra with coordinate partitions, and 
the MacWilliams identities deserve much more attention. In 
Section [VJ we review the original requirements of good linear 
codes for lossless source coding, channel coding, and lossless 
JSCC, respectively. The code-spectrum criteria of good linear 
codes for different applications are then presented. Hence, all 
the coding issues are put into one common topic: constructing 
linear codes with good spectra. By investigating the relations 
among these criteria, we realize that a good joint spectrum is 
the most important feature of a linear code. 
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